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J.  Bibb  Cain,  G.  C.  Clark,  Jr.,  and  J.  M.  Geist 

(USA) 69 

"Some  Transparent  Convolutional  Codes  with  a 
Simple  Encoder  Inverse,"  E.  Paaske  (Denmark) 

and  R.  Johannesson  (Sweden) 69 
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SESSION  C4 

RATE  DISTORTION  CODING 


Chairman.  Giuseppe  Longo,  Universita  Degll  Stud!  di  Trieste, 
Trieste  (Italy) 

"Digital  Coding  of  Analog  Waveforms,"  H.  Gish  (USA)  ....  71 


"Asymptotically  Optimal  Block  Quantization," 

A Gersho  (USA) 71 

"Differential  Encoding  for  Binary  Markov  Sources," 

G.  Thomas  (India) 72 

"Run-Length  Encoding  with  Fidelity  Criterion  for 

Binary  Markov  Sources,"  P.  K.  S.  Wah  (Switzerland) 72 

"Differential  Pulse-Code  Modulation  of  the  Wiener 

Process,"  A.  Hayashi  (Japan)  73 

"The  Simulation  Problem,"  J.  Linde  and  R.  M.  Gray 

(USA) 73 


SESSION  C5 

STOCHASTIC  PROCESSES  I 

Chairman , Robert  Boorstyn,  Polytechnic  Institute  of  N.Y., 
Brooklyn,  NY  (USA)  and  Bell  Laboratories,  Holmdel,  N.J.(USA) 


"Some  Relations  Between  Mutual  Information,  Sample 
Path  Properties,  and  Signal  Detection,"  C.  R. 

Baker  (USA) 74 

"Bounds  on  the  Mutual  Infonaation  for  Nonlinear 

Observation  Processes  with  Additive  White  Gaussian 

Noise,"  S.  Arimoto  and  T.  Hashimoto  (Japan)  74 

"Stochastic  and  Multiple  Wiener  Integrals  for 

Gaussian  Processes,"  S.  T.  Huang  and  S.  Cambanis 

(USA) 75 


"A  Time-Perturbation  of  Gaussian  Stochastic 

Processes  and  Some  Applications  to  the  Theory  of 

Signal  Detection,"  A.  Gualtierotti  (Switzerland)  75 

"On  the  Reconstruction  of  the  Covariance  of 
Stationary  Gaussian  Processes  Observed  Through 
Zero  Memory  Nonlinearities,"  S.  Cambanis  and 

E.  Masry  (USA) ; 75 

"Bounds  on  Estimation  Error  for  Gauss-Polsson 

Processes,"  A.  Segall  (Israel) 76 

"Poisson  Sampling  and  Spectral  Estimation  of 

Continuous-Time  Processes,"  E.  Masry  (USA) 76 


^ A ilr  A 
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TUESDAY  AFTERNOON,  OCTOBER  11,  2:15,  Phillips  Hall  101 


PLENARY  SESSION  C 
Invited  Lectures: 


"Some  Recent  Results  on  Characterization  of 
Measures  of  Information  Related  to  Coding," 

J.  0.  Aczel,  University  of  Waterloo, 

Waterloo,  (Canada)  77 

"Team  Decision,  Market  Signaling,  and  Information 
Theory,"  Y.  C.  Ho,  Harvard  University,  Cambridge, 

Mass.,  (USA) 77 

"Information  Theory  in  Physics,"  E.  T.  Jaynes, 

Washington  University,  St.  Louis,  Mo.,  (USA) 77 


****** 


TUESDAY  EVENING,  OCTOBER  11 

8:00  Cello  Concert  by  Nancy  Green 

Barnes  Hall  Auditorium 

****** 


9:30  Session  on  Recent  Results  and  Open  Problems 

Phillips  Hall  101 

Chairman.  David  L.  Cohn,  University  of  Notre  Dame,  Notre  Dame, 
Indiana  (USA) 

Titles  and  speakers  to  be  arranged  at  the 
Symposium. 


* * * * A ^ 


W1;dNESDAY  morning,  October  12,  9: 00,  ILR  Conference  Center 

SESSION  D1 
COMMUNICATIONS  I 

Chairman,  Jack  Keil  Wolf,  University  of  Massachusetts,  Amherst, 
Massachusetts  (USA) 

"Decoding  Random  Codes  with  an  Optimum  Threshold," 

B.  Dorsch  (Fed.  Rep.  of  Germany) 78 

"Digital  Whitening  Techniques  for  Improving  Spread 
Spectrum  Commi'nications  Performance  in  the  Presence 
of  Narrow-Band  Jamming  and  Interference,"  F,  M. 

Hsu  and  A.  Giordano  (USA) 78 
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"Recovery  of  Spread  Spectrum  Carrier  Functions," 

K.  H.  Annecke  (Fed.  Rep.  of  Germany) 79 

"Error  Probability  for  an  Incoherent  Channel  with 
Partial  Band  or  Time  Noise  and  a Mismatched  Decoding 
Statistic,"  S.  Krlch  (USA)  79 

"Adjacent  Channel  Interference  In  a Binary  Band- 
pass Communication  System,"  I.  Korn  and  M. 

Herzberg  (Israel)  80 

"Channel  Estimation  and  Decoding  In  a Multipath 
Environment,"  K.  Schneider  and  T.  P.  McCarty 

(USA) 80 

"Stochastic  Channels  as  Generalized  Communication 

Networks,"  D.  Middleton  and  J.  R.  Breton  (USA) 81 

"Prolate  Spheroidal  Wave  Functions  - The  Discrete 

Case,"  D.  Sleplan  (USA)  81 


SESSION  D2 

PATTERN  CLASSIFICATION  - FEATURES 

Chairman . K.  S.  Fu,  Purdue  University,  West  Lafayette, 
Indiana  (USA) 


"Intrinsic  Dimensionality  Spatial-Temporal  Array 

Processing,”  S.  D.  Morgera  (USA) 82 

"Nonlinear  Feature  Extraction  with  a Criterion  of 
a General  Form,"  K.  Fukunaga  and  R.  D.  Short  (USA) 82 

"An  Algorithm  for  Optimal  Nonlinear  Structure  Pre- 
serving Feature  Extraction,"  S.  Starks  and  R. 

de  Flguieredo  (USA)  83 

"Recursive  Factor  Analysis  Methods  In  Feature 

Extraction  Problems,"  L.  Kurz  and  C.  S.  Yoon  (USA) 83 

"Texture  Modeling  Using  Stochastic  Tree  Languages," 

S.  Y.  Lu  and  K.  S.  Fu  (USA) 84 

"Best  First  Parsing  of  Noisy  Waveforms,"  L.  Kanal 

and  G.  Stockman  (USA) 84 

"Syntactic  Signal  Processing,"  F.  Le  Chevalier, 

G.  Bobillot,  and  C.  Fugler-Garrel  (France) 85 

"Visual  Perception,  Invariants,  Neural  Nets," 

H.  Block,  D.  Lewis,  and  R.  Rand  (USA) 85 
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SESSION  D3 
TOPICS  IN  CODING 

Cluilrman , G.  David  Forney,  Codex  Corp.,  Newton,  Mass.  (USA) 

"Bounds  on  Che  Delay  Complexity  of  Error  Correcting 

Codes,"  Y.  Imhir  and  J.  Savage  (US.v; 86 


"Time  to  First  Error  for  an  Interleaved  Code  on  a 

Burst-Noise  Cliannel,"  R.  A.  Rutledge  (USA)  86 

"Long  Block  Codes  Can  Offer  Good  Performance," 

D.  Chase  and  11.  D.  Goldfein  (USA) 86 

"Improvements  in  Block-Retransmission  Schemes," 

J.  J.  Meczner  (USA) 87 

"Analysis  of  Merging  in  the  Bit-By-Bit  Detection/ 

Decoding  Algorithm,"  B.  D.  Pritchman  and 

J.  C.  Mixsell  (USA) 87 

"Implementation  of  Decoders  for  Block  Codes," 

E.  R.  Berlekamp  (USA) 88 

"Coding  for  Nonprobabilistic , Discrete  Channels," 

W.  L.  Root  (USA) 89 

"Chain  Coding  of  Tabular  Data  in  Noisy  Environments," 

L.  Kurz  and  C.  Mohwinkel  (USA) 89 


SESSION  D4 

UNIVERSAL  SOURCE  CODING 

Chairman , K.  M.  Gray,  Stanford  University,  Stanford, 
California  (USA) 


"Robust  Rate  Distortion  Theory, ’’  D.  Kazakos  and 
T.  Papantoni-Kazakos  (USA)  90 

"New  Results  on  Coding  of  Stationary  Nonergodic 
Sources,"  L.  D.  Davisson,  A,  Leon-Garcia,  and 

D.  L.  Neuhoff  (USA) 90 

*"Varlable-Rate  Universal  Block  Source  Coding 
Subject  to  a Fidelity  Constraint,"  K.  M. 

Mackenthun  and  M.  B.  Pursley  (USA) 90 

"Compression  of  Individual  Sequences  Via  Variable- 

Rate  Coding,"  J.  Ziv  and  A.  Lempel  (USA) 91 

*"Codlng  Theorems  for  Individual  Sequences," 

J.  Ziv  (USA)  91 


=^Denotes  long  papers. 
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"Ergodlc  and  Mixing  Properties  of  a Class  of 
Composite  Sources,"  R.  J.  Fontana  (USA)  . . 


92 


SESSION  1)5 
COMPUTATION  I 

Chairman.  Judea  Pearl,  University  of  California,  Los  Angeles, 
California  (USA) 

"An  In-Place  Self-Reordering  FFT,"  J.  K.  Beard  (USA).  ...  93 


"Synthesizing  Large  DFT's  Using  the  Mutual  Prime 

Factor  Cyclic  Algorithm,"  S.  Morgera  (USA) 93 

"Computational  Efficiency  of  Number  Theoretic 
Transform  Implemented  Finite  Impulse  Response 

Filters,"  T.  A.  Kriz  and  D.  F.  Bacliman  (USA)  93 

"Multiplicative  Convolutions  and  Fourier  Transforms," 

S.  Cohn-Sfetcu  (Canada)  94 

"The  Solution  of  Discrete  Convolutions  with  a 

Bounded  Error  Constraint,"  A.  Arcese  (Colombia)  94 

"Error  Control  in  Array  Processors,"  D.  K.  Pradhan 

(Canada)  94 

"Dual-Mode  Logic:  A New  Redundancy  Technique  for 
tlie  Design  of  Easily  Tested  Logic  Circuits,"  S. 

DasGupta,  C.  Hartmann,  and  L.  Rudolph  (USA)  94 


****** 


WEDNESDAY  AFTERNOON,  OCTOBER  12,  2:00,  ILR  Conference  Center 
SESSION  El 

TOPICS  IN  ALGEBRiMC  CODES 

Chairman.  R.  T.  Chlen,  University  of  Illinois  at  Urbana-Champalgn 
Urbana,  Illinois  (USA) 


"Group  Theoretic  Codes  for  B.)nary  Asymmetric 

Channels,"  S,  Constantin  and  T.  R,  N.  Rao  (USA) 96 

"Some  Problems  on  Permutation  Group  Codes," 

G.  Cohen  and  M.  Deza  (France)  96 

"Configuration  Matrices  of  Croup  Codes  for  the 

Gaussian  Channel,"  E.  Bigllerl  and  M.  Ella  (Italy) 97 

"An  Algorithm  for  Coding  Undirected  Graphs," 

K.  Venkataraman  and  K.  Thulasiraman  (India)  97 


"New  Classes  of  Sequences  with  Good  Correlation 

Properties,"  D.  A.  Shedd  and  D.  V.  Sarwate  (USA) 97 

"On  Linearization  of  Nonlinear  Combinations  of 
Linear  Shift  Register  Sequences,"  T.  Herlestam 
(Sweden) 98 

"Recent  Results  on  Algebraic  Soft-Decision 
Decoding,"  T.  Hwang  (USA),  N.  Due  (Australia), 

C.  Hartmann  (USA),  and  L.  Rudolph  (USA)  98 

SESSION  E2 

STOCHASTIC  PROCESSES  II 

Chairman , Charles  Baker,  University  of  North  Carolina,  Chapel 
Hill,  North  Carolina  (USA) 

"Optimal  Sampling  of  Independent  Increment 

Processes,"  B.  Stuck  and  C.  Newman  (USA) 100 

"A  Prediction  Problem,"  D.  Slepian  (USA) 

"Properties  and  Applications  of  a Useful  Class 
of  Two-Dimensional  Random  Fields,"  R.  W.  Fries 
and  J.  W,  Modestino  (USA) 100 

"Extrapolation  of  Homogeneous  Random  Fields," 

H.  Ogura  (Japan) 100 

"Envelope  Constrained  Filters  with  Uncertain 

Input,"  R.  J.  Evans  and  A.  Cantonl  (Australia) 101 

"Moment  Formulas  for  the  Number  of  Slope- 
Constrained  Level  Crossings  by  a Wide  Class 
of  Stochastic  Processes,"  D.  R.  Anderson  and 

D.  D.  Carpenter  (USA) 101 

SESSION  E3 
SPEECH 

Chairman , Lallt  Bahl,  IBM  Research,  Yorktown  Heights,  N.Y.  (USA) 


"Tree  Encoding  of  Speech  at  8000  BPS,"  S.  G.  Wilson 

(USA) 103 

"Tree  Encoding  of  Speech  Based  on  Short  Term 

Autocorrelations,"  S.  Mattsson  (Sweden)  103 


"Stack  Algorithm  Speech  Encoding,"  S.  Mohan  and 
J.  B.  Anderson  (Canada)  


104 


"Soft  Decision  Demodulation  for  PCM  Encoded  Speech 

Signals,"  C.  E.  Sundberg  (Sweden)  104 

"Mathematical  Treatment  of  Speech  Signals,"  D.  Wolf 

and  H,  Brehm  (Fed.  Rep.  of  Germany) 105 

"Recognition  of  Words  from  a Regular  Language  in 

the  Presence  of  Noise,"  R.  L.  Kashyap  (USA) 105 

"Estimation  of  Phonemic  Confusion  Using  Mahalanobis 

Distance,"  S.  Tazaki,  H.  Osawa,  Y.  Yamada,  and 

T.  Gotoda  (Japan) 106 


SESSION  E4 

CHANNEL  CODING  THEORY 

Chairman , S.  Arimoto,  Osaka  University,  Osaka  (Japan) 


"Shannon's  'Proof  of  the  Noisy  Coding  Theorem," 

J.  L.  Massey  (USA)  107 

*"A  New  Look  at  Exponential  Error  Bounds  for  Memory- 
less Channels,"  I.  Cslszar,  J.  Korner,  and  K.  tiarton 
(Hungary) 107 

"Reliability  Function  of  a DMC  at  Rates  Above 

Capacity,"  J.  Korner  (Hungary) 107 


"Block  Coding  for  Discrete  Stationary  d-Continuous 

Noisy  Channels,"  R.  M.  Gray  and  D.  S.  Ornstein  (USA).  . . . 108 


"Randomness  in  Discrete  Channels  with  Memory," 

P.  C.  Shields  and  D.  L.  Neuhoff  (USA)  . . .” 109 

"iietrics  Matched  to  the  Discrete  Memoryless  Channel," 

G.  Seguln  (Canada) 109 

"The  Geometry  of  Probability  Space,"  R.  E.  Blahut 

(USA) 109 


SESSION  E5 
COMPUTATION  II 

Chairman.  T.  Kallath,  Stanford  University,  Stanford, 
California  (USA) 


"Bounds  on  Update  Algorithms,"  R.  Flower  (USA)  110 

"Stochastic  Error  Analysis  of  Spline  Approximation," 

H.  Welnert  (USA),  G.  Sldhu  (Mexico),  and  R.  Byrd 

(USA) 110 


^Denotes  long  papers. 
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"Fitting  Curves  to  Deterministic  Data,"  H.  Welnert 


(USA),  R.  Byrd  (USA),  and  G.  Sidhu  (Mexico) 110 

Square-Root  Algorithms  for  Parallel  Processing 
in  Optimal  Estimation,"  M.  Morf,  J.  Dobbins, 

B.  Friedlander,  and  T.  Kailath  (USA) Ill 

"Ladder  Forms  for  Estimation  and  Detection," 

M.  Morf,  D.  Lee,  and  A.  Vieira  (USA) Ill 

"On  Some  Complexity  Problems  in  Communication 

Theory,"  K.  Yao  (USA) 112 

"On  the  Use  of  Associative  Memories  for  Pattern 
Recognition  and  Information  Processing," 

Y.  H.  Pao  and  W.  L.  Schultz  (USA) 112 


****** 


WEDNESDAY  EVENING,  OCTOBER  12,  8:00,  Phillips  Hall  101 

"Current  Concerns  of  IEEE:  A Discussion"  by  Irwin 
Feerst  and  Ivan  Getting,  Candidates  for  President 
of  IEEE.  Moderated  by  F.  Jelinek  


* * ^ * 


THURSDAY  ^^0RNING,  OCTOBER  13,  8:00,  ILR  Conference  Center 
Movie  on  ESP  Experments 


A A A A A 


THURSDAY  MORNING,  OCTOBER  13,  9:00,  ILR  Conference  Center 

SESSION  FI 
COMMUNICATIONS  II 

Chairman,  Robert  Price,  Sperry  Research  Center,  Sudbury, 
Massachusetts  (USA) 

"Maximum  Posterior  Probability  Demodulation  of  Angle- 
Modulated  Signals,"  D.  W.  Tufts  and  J.  T.  Francis  (USA)  . . 113 

"Pre-Processing  of  Signals  for  a Class  of  CDM  Parallel 

Data  Transmission  Systems,"  F.  Ghanl  (India) 113 
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"Error  Probability  for  Zero-Crossing  Detection  of 

Digital  FM,"  H.  Abut  (Turkey) 113 

"Optimum  PCM  Codes  for  DPSK,"  C.  E.  Sundberg 

(Sweden) 114 

"COSYDAl  - A New  Integrated  Communication  System  with 
Data  Compression  and  Error  Control,"  M.  Barducci, 

G.  Benelli,  V.  Cappellini,  and  E.  Del  Re  (Italy)  114 

"Error  Rate  Performance  of  a Fading  Multichannel 

System,"  H.  E.  Nichols  (USA)  115 

"Trellis  Coding  with  Expanded  Channel-Signal  Sets," 

G.  Ungerboeck  (Switzerland)  115 


SESSION  F2 
DETECTION  II 

Chairman.  Michael  B.  Pursley  , University  of  Illinois  at  Urbana  - 
Champaign,  Urbana,  Illinois  (USA) 

"Some  Statistical  Properties  of  the  Monopulse  Ratio," 

I.  Ranter  (USA) 116 

"Detection  and  Parameter  Estimation  of  Closely  Spaced 
Multiple  Targets,"  U.  Nickel  (Fed.  Rep.  of  Germany)  ....  116 


"Optimal  Detection  of  a Two-State  Markov  Process  in 

Noise,"  E.  Panayircl  (Turkey)  116 

"Digital  Signal  Detection  with  Quantized  Observations," 

S.  Reisenfeld  (USA)  117 

"Digital  Detection  of  Periodic  Signals  in  Gaussian 

Noise,"  M.  Blanco  (USA)  118 


"Detection  of  Weak  Signals  In  Narrowband  Non- 

Gaussian  Noise,"  J.  W.  Modestino  and  A.  Nlngo  (USA)  ....  118 


"Leading  Edge  Estimation  Errors,"  1.  Bar-David 

and  D.  Anaton  (Israel)  119 

"Multi-Modal  Estimators  of  Pulse  Parameters  for 
Threshold-Extension  Demodulators,"  I.  Bar-David 
and  B.  G.  Goldberg  (Israel) 119 


SESSION  F3 
NETWORK  AND  QUEUES 

Chairman , David  Sleplan,  Bell  Telephone  Laboratories, 
Murray  Hill,  N.J.  (USA) 
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"Extension  of  an  Adaptive  Distributed  Routing 

Algorithm  to  Mixed  Media  Networks,"  A.  Ephremides 

(USA) 120 

"Analysis  of  an  Adaptive  Routing  Strategy  for 

Computer  Communication  Networks,"  T.  S.  Yum  and 

M.  Schwartz  (USA) 120 

"Optimal  Routing  for  Line-Switched  Data  Networks 

Using  Distributed  Computation,"  A.  Segall  (USA)  121 

"Performance  of  Queueing  Systems,"  M.  Nguyen  and 

R.  Pickholtz  (USA) 121 

"On  A Stochastic  Integral  Equation  Model  for  Markov 

Queueing  Networks,"  F.  J.  Beutler  (USA)  121 

"Theoretical  Upper  Bounds  on  Spectral-Spatial 
Utilization  in  a Cellular  Land  Mobile  Communica- 
tions Systems,"  G.  R.  Cooper  and  R.  W.  Nettleton 
(USA) 122 

"A  Protocol  for  Resolving  Conflicts  on  Aloha 

Channels,"  J.  Capetanakls  (USA)  122 

"Integrated  Random-Access  Reservation  Schemes 
for  Multi-Access  Communications  Channels," 

I.  Rubin  (USA) 123 

"Multiplicative  Multiple-Access  Method  for  the 
Inquiry/Response  Channel,"  J.  Bhullar  and 

J.  B.  Anderson  (Canada) 123 


SESSION  F4 

RATE  DISl’ORTION  THEORY 

Chairman . Janos  Ko'rner,  Mathematical  Institute  of  the  Hungarian 
Academy  of  Sciences,  Budapest  (Hungary) 

"New  Bounds  to  R(D)  of  a Source  Whose  Output  is  the 


Sum  of  Two  Independent  Random  Entities"  D. 

Anastassiou  and  U.  Sakrlson  (USA)  125 

"Rate-Distortion  Functions  for  Continuous  Alphabet 

Memoryless  Sources,"  S.  L.  Fix  and  D.  L.  Neuhoff 

(USA) 125 

"On  the  Asymptotic  Behavior  of  the  Rate  Distortion 

Function,"  G.  Cohen  (France)  and  C.  L.  Simionescu 

(Romania) 125 

"An  Approximation  to  Rate  Distortion,"  G.  Cohen 

(France)  and  C.  L.  Simionescu  (Romania)  126 
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"Bounds  on  the  Performance  of  Linear  Source 

Coding,"  T.  C.  Ancheta  (USA) 126 

"Joint  Source-Channel  Time-Invariant  Trellis 

Encoding,"  J.  G.  Dunham  and  R.  M.  Gray  (USA) 126 

"Tree  Sequential  Encoding  for  Sources  with  Memory 

Under  a Fidelity  Criterion,"  H.  H.  Tan  (USA) 127 

"Causal  Rate-Distortion  Theory  for  Real-Time 

Estimation,"  J.  L.  Center  (USA)  127 


SESSION  F5 

FOUNDATIONS  AND  INNOVATIONS 
(Special  Session) 

Chairman,  Terrence  Fine,  Cornell  University,  Ithaca,  N.Y.  (USA) 


"Optimizing  Receivers  for  Remote  Perception  Using 

Sensory  Noise  Reduction  Techniques,"  C.  Honorton  (USA)  . . 128 


+"Axiomatization  and  Representation  of  Qualitative 
Information,"  Z.  Domotor  (USA) 128 

"Information  Theory  and  Organization  Theory," 

R.  F.  Drenick  (USA) 128 

f'Recent  Work  on  Algorithmic  Information  Theory," 

G.  Chaitin  (USA)  129 

fComplexlty-Based  Induction  Systems:  Comparisons 
and  Convergence  Theorems,"  R.  Solomonoff  (USA) 129 


^ -k  ■k  it  ^ ^ 


THURSDAY  AFTERNOON,  OCTOBER  13,  1:30,  ILR  Conference  Center 

SESSION  G1 
ALGEBRAIC  CODES 

Chairman,  I.  S.  Reed,  University  of  Southern  California,  Los 
Angeles,  California  (USA) 

"Syndrome  Decoding  of  Binary,  Rate  - k/n  Convolutional 
Codes,"  J.  P.  H.  Schalkwijk,  A.  J.  Vinck,  and  K.  A. 


Post  (Netherlands) 130 

"Concatenated  Codes  for  Improved  Performance  with 
Applications  to  the  Rayleigh  Fading  Channel," 

J.  Pleper,  J.  Proakis,  R.  Reed,  and  J*  Wolf  (USA)  130 


tUenotes  invited  paper. 
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"Continued  Fractions  and  lier lekamp ' s Algorithm," 

L.  R.  Welch  and  R.  A.  Scholtz  (USA) 130 

"A  Decoding  Procedure  for  Multiple-Error-Correcting 

Cyclic  AN  Codes,"  Wen-Yung  Yeh  (USA) 131 

"Arithmetic  Codes  in  Residue  Number  Systems," 

F.  Bars!  and  P.  Maestrinl  (Italy) 131 

SESSION  G2 
IDENTIFICATION 


Cha  rman,  Emanuel  Parzen,  State  University  of  New  York  at 
Buffalo,  Amherst,  New  York  (USA) 


"A  Note  on  the  Identification  of  Two  Dimensional 

AR>b\  Models,"  H.  W.  Penm  and  M.  Kanefsky  (USA) 132 

*"The  Recursive  Identification  of  Stochastic  Systems 
Using  an  Automaton  with  Slowly  Growing  Memory," 

B.  D.  Kurtz  and  P.  E.  Caines  (Canada) 132 

"MLE  of  Density  Forms  and  Identity  of  Normal  Mixtures," 

D.  S.  Arantes  (Brazil)  133 

"Consistent  Estimation  of  the  Order  of  Autoregressive 
Systems,"  W.  Hwang  (USA)  133 

"Resolution  Enhancement  in  the  Autoregressive 

Spectral  Estimation  of  Noisy  Signals,"  H.  Kaveh 

(USA) 133 


SESSION  G3 

MATHEMATICS  IN  INFORMATION  THEORY 
(Invited  Session) 

Organizer  and  Chairman,  Paul  Shields,  University  of 


Toledo,  Toledo,  Ohio  (USA) 

"Pseudo-Random  Numbers,"  H.  G.  Niederreiter  (USA)  135 
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MULTIPLE  TERMINAL  CHANNEL  THEORY  - Thomas  M.  Cover 
(Information  Systems  Laboratory,  Stanford  University, 
Stanford,  CA  94305).  A unified  presentation  of  new 
or  recent  results  in  multiple  user  channel  theory 
will  be  given.  In  particular  the  duality  between 
the  multiple  access  channel  capacity  (Ahlswede,  Liao) 
and  multiple  source  compression  capacity  (Slepian, 

Wolf)  will  be  developed.  Repeated  use  of  the  asymptotic 
equlpartition  property  will  be  seen  to  yield  simple 
proofs  for  achievable  rate  regions  for  certain  broadcast 
channels,  multiple  access  channels,  two-way  channels, 
relay  channels,  and  the  above-mentioned  channels  with 
feedback . 

At  this  time  no  theoretical  evidence  contradicts  the 
hope  that  we  shall  eventually  have  an  information  - 
theoretic  characterization  of  the  capacity  region  for 
an  arbitrary  information  network  with  m senders,  n 
receivers,  and  arbitrary  feedback  loops. 
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SOME  RESULTS  ON  A GENERAI.  CLASS  OF  QUASICYCLIC  CODES,  Pierre  A.  von 
Kaenel  (Department  of  Mathematics  and  Computer  Science,  The  Univer- 
sity of  Nebraska  at  Omaha,  Omaha,  Nebraska  68101).  A general  class 
of  quasicyclic  (Q-C)  codes  which  contains  many  of  the  special  cases 
investigated  In  the  literature  such  as  (2n,n)  Q-C  codes  will  be  de- 
fined. An  algebraic  method  for  designing  the  subclass  of  these 
codes  having  length  2 (2  blocks  of  n-tuples)  will  then  be  presented. 
Using  this  method,  the  first  rows  of  the  circulant  matrices  contained 
In  the  generator  matrix  of  a Q-C  code  In  this  subclass  can  be 
selected  In  such  a way  that  the  lower  bound  on  minimum  distance  is 
considerably  improved  over  that  of  an  arbitrary  Q-C  code  of  the  same 
size.  Chen  et  al.  have  found  general  lower  bounds  on  minimum  distance 
for  (2n,n)  Q-C  codes;  however  our  designing  method  can  improve  this 
bound  by  as  much  as  a factor  of  two.  Other  classes  of  Q-C  codes  can 
be  designed,  and  some  codes  can  be  found  which  compare  favorably 
with  the  best  known  linear  codes  of  that  size.  For  example,  a (42, 

11)  Q-C  code  can  be  constructed  with  a minimum  distance  d bounded  by 
14  < d <16.  The  best  known  (42,11)  linear  code  has  d = 15. 


SOME  RESULTS  ON  THE  STRUCTURE  OF  GOPPA  CODES,  C.L.  Chen  (D18/707, 
P.O.  Box  390,  IBM  Corporation,  Poughkeepsie,  New  York  12602).  A 
relationship  between  Goppa  codes  and  cyclic  codes  is  established. 
Goppa  codes  are  identified  as  shortened  cyclic  codes.  Irreducible 
Goppa  codes  are  classified  Into  equivalence  classes  to  reduce  the 
complexity  of  the  weight  enumeration  of  the  codes.  The  location 
points  of  a code  word  of  an  Irreducible  Goppa  code  are  related  to 
the  points  of  a finite  projective  geometry.  The  Irreducible  Goppa 
codes  extended  with  an  overall  parity  check  are  Investigated.  All 
binary  extended  Irreducible  codes  of  a fixed  length  with  minimum 
distance  8 are  shown  to  have  the  same  weight  distribution. 


AN  ENUMERATION  OF  SELF-DUAL  CODES,  J.H.  Conway  (University  of 
Cambridge,  Cambridge,  England)  and  Vera  Pless  (University  of  Illinois 
at  Chicago  Circle,  Chicago,  Illinois  60680).  A linear  code  C is 
called  self-dual  if  C - c*-.  Self-dual  codes  are  a practical  and 
Interesting  class  of  codes  because  they  Include  some  of  the  best 
algebraic  codes  like  the  Golay  codes,  the  quadratic  residue  codes, 
and  the  symmetry  codes  and  also  because  more  Is  known  theoretically 
about  them,  l.e.,  their  weight  distributions  are  combinations  of 
Gleason  polynomials.  Further,  it  is  known  that  they  satisfy  a 
Varshamov-Gllbert  bound. 
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A binary  self-dual  code  must  have  all  weights  even.  If  in  addition 
all  weights  are  divisible  by  4,  the  code  is  called  doubly-even.  The 
Golay  (24,12)  code  is  doubly-even  as  is  the  Hamming  (8,4)  code.  This 
paper  completely  classifies  all  (32,16)  double-even  codes. 

14 

It  is  shown  that  of  the  2*3*5‘  •••(2  +1)  self-dual  doubly-even  codes 

there  are  85  inequivalent  such  codes,  five  of  which  have  minimum 
weight  8,  three  new  codes  and  the  two  known  codes;  a quadratic 
residue  code  and  a Reed-Muller  code.  For  each  code  in  an  equivalence 
class,  a canonical  basis  is  given,  as  is  the  entire  group  of  the  code 
and  its  weight  distribution.  From  this  list  we  are  able  to  identify 
all  self-dual  (n,n/2)  codes  for  n < 32.  The  data  for  n « 30,  28, 
and  26  is  new.  Further,  we  show  that  for  the  next  case  (40,20),  the 
number  of  Inequivalent  double-even  codes  is  greater  than  17,000  so 
that  it  is  not  possible  to  classify  them  with  present  or  forseeable 
methods . 


CODES  OVER  GF(4)  AND  COMPLEX  LATTICES,  N.J.A.  Sloane  (Bell  Labs, 
Murray  Hill,  N.J.).  This  paper  studies  the  relationship  between 
error-correcting  codes  over  GF(4)  and  complex  lattices.  The  theta- 
functions  of  self-dual  lattices  are  characterized.  Two  general 
methods  are  presented  for  constructing  lattices  from  codes.  Several 
examples  are  given.  Including  a new  lattice  sphere-packing  in  R3f>. 


CODES  WITH  REGULAR  GAPS  IN  THEIR  WEIGHT  DISTRIBUTION,  V.K.  Bhargava, 
(Department  of  Electrical  Engineering,  Concordia  University,  Montreal, 
Canada  H3G  1M8) . Consider  (mn,m)  rate  1/n  quasl-cyclic  codes  whose 
generator  matrices  in  the  partitioned  form  are  given  by  G • [!„,  Ci, 
C2,...,  Cn-l].  By  choosing  the  clrculant  matrices  C^’s  judiciously, 
we  have  discovered  a number  of  new  rate  1/n  quasl-cyclic  codes  with 
regular  gaps  in  their  weight  distribution.  These,  among  others. 
Include  a (26,13)  d • 5 code  with  weights  £ 1 or  0 (mod  4),  a (52,13) 
d » 15  code  with  weights  = 3 or  0 (mod  4),  a (54,18)  d - 12  code  with 
weight  = 0 (mod  4)  and  a family  of  (3p,p)  codes  with  weights  = 3 or  0 
(mod  4).  We  conclude  with  a number  of  rate  1/2  and  rate  1/3  quasi- 
cycllc  codes  showing  regular  gaps  in  their  weight  distribution. 
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A NEW  CLASS  OF  ASYMPTOTICALLY  GOOD  CODES  BEYOND  THE  ZYABLOV  BOUND, 
Yasuo  Suglyama  (Mitsubishi  Electric  Corporation,  80  Nakano, 
Minamishimizu , Amagasaki,  Hyogo  661,  Japan),  Masao  Kasahara  (Depart- 
ment of  Communication  Engineering,  Osaka  University,  Yamada-Kami, 
Suita,  Osaka  565,  Japan),  Shigeichi  Hirasawa  (Mitsubishi  Electric 
Corporation,  1-1-2  Wadasakicho,  Hyogoku,  Kobe,  Hyogo  652,  Japan) 
and  Toshihiko  Namekawa  (Department  of  Communication  Engineering, 

Osaka  University,  Yamada-Kami,  Suita,  Osaka  565,  Japan).  We  con- 
struct a new  class  of  asymptotically  good  codes.  The  asymptotically 
good  codes  are  constructed  by  Iteratively  superimposing  Justesen 
codes  on  a Justesen  code.  The  codes  can  be  regarded  as  members  of 
a class  of  generalized  concatenated  codes.  For  any  rate  r,  0 <r<l, 
the  ratio  d/n  of  minimum  distance  d to  length  n of  the  codes  satisfies 
the  relation. 


lim  inf  (d/n)  ^ 

n-Mu 


(1-2'S-r 

— j — 

j-1  [1/(0^J^  2^)) 


h"^  (1/2) 


where  )!,  is  a positive  integer,  H(*)  is  the  binary  entropy  function, 
and  9(J)  = H“^  <1/2J)/H“i  (1/2)  for  J-l,2,...,£.  The  lower  bound 
on  the  ratio  d/n  for  the  new  codes  lies  above  the  Zyablov  bound  for 
rate  r,  0.31  <r<l,  while  the  lower  bound  on  the  ratio  d/n  for  all 
asymptotically  good  codes  so  far  known  to  us  lies  below  or  on  the 
Zyablov  bound. 


COMPUTER  DETERMINATION  OF  THE  TRUE  MINIMUM  DISTANCE  OF  ALL  BINARY 
CYCLIC  CODES  OF  ODD  LENGTHS  FROM  69  to  99,  G.  Promhouse  and  S.E. 
Tavares  (Department  of  Electrical  Engineering,  Queen's  University, 
Kingston,  Ontario,  Canada).  A computer  search  has  been  made  to 
determine  the  true  minimum  distance  d for  all  binary  cyclic  codes 
having  odd  lengths  n in  the  range  69  ^ n s 99.  Using  an  algorithm 
originally  developed  by  C.L.  Chen,  the  generator  matrix  of  each 
(n,k)  binary  cyclic  code  was  determined  and  put  in  systematic  form 
G.  All  possible  code  words  obtained  by  summing  the  rows  of  G,  1 at 
a time,  i “ l,2,...,v,  were  examined  and  the  minimum  distance  dv  of 
this  set  was  recorded.  Then  d “ dv  whenever  v > { (dv-l)k/n  } - 1. 
Like  Chen,  known  equivalences  among  cyclic  codes  were  taken  into 
account  and  only  one  code  from  each  equivalence  class  was  listed. 

Let  g(x)  divide  (x*^-l),  where  (x-1)  is  not  a factor  of  g(x).  Then 
the  minimum  distance  of  the  codes  generated  by  g(x),  (x-1)  g(x)  and 
cheir  respective  duals  were  listed  together.  For  each  such  pair  of 
codes  in  the  table,  the  value  of  v for  which  a code  word  of  minimum 
weight  first  appeared  is  listed.  The  codes  found  were  compared  with 
the  list  of  best  codes  tabulated  by  Sloane.  Many  good  cyclic  codes 
have  been  found.  Among  the  best  (n,k,d)  cyclic  codes  found  are  the 
following:  (73,36,16),  (73,27,20),  (85,20,28),  (89.56,11),  (91,51, 
14),  (91,28,24),  (93,23,29). 
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LINEAR  RECURSIVE  CODES,  Michael  Willett  (Mathematics  Department, 
University  of  North  Carolina  at  Greensboro).  Let  C be  an  (N,K) 
cyclic  code  over  F » GF(pe)  with  parity-check  polynomial  h(x)  » 
h2^(x)h2(x) . . .ht(x) , where  the  hjCx)  are  distinct  and  irreducible 
over  F.  It  is  well-known  that  C equals  G(h),  the  vector  space  of 

N-1 

all  N-tuples  u - (u  ) satisfying  h(E)u  “ 0 with  Eu  • u . , and 
no  n n n+l 

all  subscripts  reduced  modulo  N.  Linear  recursion  theory  is  used 

to  show  that  it  is  sometimes  possible  to  choose  s » (kj^,  k2,...,  » 

k.  s 1,  K - Ek.  deg  (h  ) < N,  in  such  a way  that  the  (N,K  ) (non- 
^ ^ ^ k k k 

cyclic)  linear  code  G(h  ),  h (x)  “ h^^  (x)  h2  (x)...h^  (x) , has  the 
same  minimum  distance  as  the  subcode  G(h). 
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CONVERGENCE  BOUNDS  OF  BAYES  LEARNING  ESTIMATORS,  Ary  Kiselsteln  and 
Amnon  Hammer  (Department  of  Electrical  Engineering,  Technlon  - Israel 
Institute  of  Technology,  Haifa,  Israel).  The  paper  analyzes  the  con- 
vergence of  signal  estimation  by  means  of  Bayes  learning.  The  true 
signal  m^  Is  assumed  as  deterministic,  fixed,  and  embedded  In  additive 
Gaussian  noise  N(0,o2).  quality  of  convergence  depends  on  the 

mean  m^,  and  variance  of  of  the  Initially  assumed  signal  proba- 
bility density.  The  estimator  performance  Is  evaluated  by  means  of 

the  estimate  variance  value  of  is  of  course  minimal  If 

mo  “ m^.  When  mg  Is  chosen  as  mo  * m^  ± 6,  the  optimal  choice  for 

2 2 2 ~~1 
Ip  becomes  ip  “ 8 , yielding  a bound  for  e as 
o o n 


^ n I ^2  2 

6“const.  o + n 8 


The  value  of  Cn  does  not  converge  down  monotonlcally , with  Increase 

of  n,  only  If  ♦ ^ > 26^. 

o 

~2 

The  number  of  learning  Intervals  required  to  decrease  e jj  to  a frac- 

1 ^2  ^ 

tlon  6 of  Its  Initial  value  Is  given  by  2 ~ ’ 

8 

It  Is  shown  that  the  Cramer-Rao  bound  exceeds  the  Bayes  learning 

2 2 

estimator  bound  when  4i  “ 6 . The  paper  Is  concluded  with  numerical 
examples. 


OPTIMAL  BAYES  DECISION  RULES  IN  MULTI  CATEGORY  CLASSIFICATION 
PROBLEMS  WITH  TIME  SERIES,  R.L.  Kashyap  (School  of  Electrical  Engi- 
neering, Purdue  University,  West  Lafayette,  Indiana  47907).  We 
consider  the  problem  of  classifying  a given  time  series  Zj^  - {y(l) , 
...,y(N)}  Into  one  of  r classes  Cj,  1-1,..., r.  The  stochastic 
process  y(’)  Is  assumed  to  obey  an  autoregressive  structure  Involv- 
ing a parameter  vector  £,  whose  probability  density  p(£|Cj)  depends 
on  the  class  to  which  Z or  y(*)  belongs.  Assuming  appropriate  ex- 
pressions for  p(0,|Cj),  we  show  that  the  probability  density  of 
characterizing  each  class,  namely  p(Zfj|Ci),  possesses  a vector  T of 
sufficient  statistics  l.e. , all  the  Information  about  Zfj  needed  for 
the  discrimination  between  the  various  classes  Is  contained  In  the 
vector  £ " (Fj  (Z(]) , . . . (Z^))^,  where  the  functions  (Zj^) , 

nrfl  have  the  same  structure  f^r  all  N.  Thus  the  best  pos- 
sible feature  set  for  the  problem  Is  We  deduce  the  optimal 
decision  rule  which  minimizes  the  average  probability  of  error.  We 
will  compare  the  optimal  features  and  the  corresponding  optimal 
decision  rule  with  other  feature  sets  and  decision  rules  mentioned 
In  the  literature. 
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ON  AN  ADMISSIBLE  LINEAR  DISCRIMINATION  RULE  FOR  MULTICLASS  PATTERN 
RECOGNITION,  Dimitri  Kazakos  and  Basile  Dimitriadis  (Department  of 
Electrical  Engineering,  State  University  of  New  York  at  Buffalo, 
Amherst,  New  York  14260).  Let  X be  an  n-dlmensional  observation 
which  we  wish  to  classify  according  to  the  linear  rule: 

decide  if  C^_^  < x'^V  5 C^ , i - 1 , . . . , m. 

Let  (Mj^.Rj)  be  the  mean  and  covariance  matrix  of  X under  hypothesis 
Hi,  1 1...,  m,  m 2:  3. 

Let  be  the  probability  of  correct  classification  under  Hj, 

where : 

D = {V,  -»>  = C < C,  <...<  C , < C = +oo} 
o 1 m- 1 m 

A rule  D*  is  said  to  be  admissible  if  there  is  no  other  rule  Di  for 
which 

Pdi(Di)  ^ ^ ^ 

In  the  present  paper  we  show  that,  under  mild  conditions  on  the 
probability  density  functions  {f(X|H^)},  the  class  of  admissible 
rules  D*  belongs  to  the  set; 

{V,  [aR^  + bRj  + cRj^]  V - d(M^-Mj)  + e(Mj-M^),  1 , j ,ke  { 1 , . . . , m), 
a,b,c  2 0} 

(the  constants  are  given  in  terms  of  V for  the  admissible  class). 

OPTIMIZATION  OF  ADD-ON  SIGNALS  BY  MEANS  OF  A MODIFIED  TRAINING 
ALGORITHM  FOR  LINEAR  CLASSIFIERS,  Wolfram  Szepanski  (Institute  fur 
Elektrlsche  Nachrlchtentechnik,  Rheinlsch-Westfalische  Technische 
Hochschule  Aachen,  D-5100  Aachen,  Alte  MaastrlcLter  Str.  23, 

Germany).  In  add-on  data  transmission  for  TV  channels,  a data  sig- 
nal is  added  to  the  active  parts  of  the  video  signal  in  order  to 
transmit  additional  information. 

Compatibility  requirements  and  their  Impact  on  the  selection  of 
suitable  data  signals  are  discussed.  To  make  maximum  use  of  the 
correlation  within  the  video  signal,  the  data  and  the  video  signals 
are  considered  to  be  functions  of  three  variables  x,  y,  and  t. 

Since  correlation  reception  of  these  three-dimensional  signals 
corresponds  to  a linear  classification  problem  in  pattern  recogni- 
tion, a modified  training  procedure  for  linear  classifiers  is  used 
to  optimize  the  data  signals  for  minimum  error  probability. 
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The  results  of  the  optimization  procedure  are  presented  together 
with  achievable  data  rates  and  error  probabilities  for  this  type 
of  data  transmission  over  TV  channels. 


QUANTIZATION  COMPLEXITY  AND  TRAINING  SAMPLE  SIZE  IN  PATTERN  RECOGNI- 
TION, Dimitri  Kazakos  (Electrical  Engineering  Department,  State 
University  of  New  York  at  Buffalo,  Amherst,  New  York  14260).  For 
the  k-hypothesls  detection  problem,  it  is  shown  that  among  the 
k-classes  of  probability  density  functions  with  m fixed  quantiles 
the  histograms  achieve  the  least  favorable  performance,  as  it  is 
measured  by  the  probability  of  correct  detection  and  Chernoff 
distance. 

We  assume,  then,  that  the  m cell  probabilities  are  estimated  using 
n training  samples  per  class.  Using  the  estimated  cell  probabilities 
new  observations  are  processed.  A distribution-free  upper  bound  to 
the  probability  of  c-deviatlon  between  the  actual  probability  of 
correct  detection  and  the  theoretical  (known  quantities)  probability 
is  derived,  given  as  a function  of  (m,n,e ,k,UQ) , where  Uq  is  uniform 
upper  bound  to  the  true  class  densities.  The  bound  converges  ex- 
ponentially to  zero  as  n -*■  “>.  Exponential  convergence  is  maintained 
by  choosing  ra  = n“,  0 < a < 1.  Hence,  the  rule  ra  = n®  answers  the 
long  standing  question  of  relating  m and  n in  a distribution-free 
manner.  The  question  of  optimal  choice  of  a is  also  discussed. 


ERROR  ESTIMATION  AND  SAMPLE  DESIGN,  Stephen  J.  Whitsitt  (Data  Systems 
Department,  TRW  DSSG,  One  Space  Park,  Bldg.  90,  Room  2200,  Redondo 
Beach,  California  90278)  and  David  A.  Landgrebe  (Laboratory  for 
Applications  of  Remote  Sensing,  Purdue  Industrial  Research  Park, 

1220  Potter  Drive,  W,  Lafayette,  Indiana  47906),  An  estimator  for 
error  which  employs  both  posterior  probabilities  and  sample  stratifi- 
cation by  class  is  developed  as  an  extension,  and  completion  of  a 
family  of  count  and  posterior  estimators.  The  effectiveness  of  this 
estimator  is  demonstrated  using  experiments  involving  normal  patterns 
Applications  in  testing  fixed  classifiers  are  discussed. 


DISTRIBUTION-FREE  PROBABILITY  INEQUALITIES  FOR  THE  DELETED  AND  THE 
HOLDOUT  ERROR  ESTIMATES,  L.P.  Devroye  and  T.J.  Wagner  (University 
of  Texas,  Austin,  Texas  78712).  Let  (Xj  , Bj^) , . . . , (X^, , 0^)  ^ 

sequence  of  Independent  IRd  X IR  -valued  random  vectors  distributed 
as  (X,0),  A discrimination  rule  0,  a real-valued  function  of  x and 
the  data.  Induces  a probability  of  error  Lj^  = P'O  ^ e|  (Xj^ , 01^)  , . . . , 
(Xf,,0n)J'  To  estimate  L^^  from  6 and  the  data,  the  resubstltut  ion, 
deleted  and  holdout  estimates  can  he  u.scd. 
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For  rules  that  use  X and  some  or  all  of  the  k nearest  neighbors  to 
X (in  an  arbitrary  way),  upper  bounds  are  derived  for  P{  iTn  - L^l 
> e}  where  is  one  of  the  said  estimates.  The  bounds  are 
distribution-free,  and  depend  only  upon  n,  k and  e.  Exponential 
bounds  exist  that  also  depend  upon  d.  The  inequalities  obtained 
for  the  deleted  estimate  are  the  tightest. 


USE  OF  MAJORITY-VOTE  IN  n-SAMPLE  DECISION  MAKING,  Sargur  N.  Srlhari 
(Computer  Science  Department,  Wayne  State  University,  Detroit,  MI 
A8202).  The  use  of  context  in  pattern  recognition  is  achieved  by 
the  application  of  compound  decision  rules.  The  n-sample  decision 
problem  is  a special  case  of  the  compound  decision  problem  where 
each  of  n samples  is  known  to  belong  to  the  same  but  unknown  state 
of  nature.  A natural  method  of  utilizing  a simple  decision  rule  in 
determining  the  true  state  of  nature  is  to  classify  each  sample 
individually  and  choose  the  class  represented  most  often  among  n 
decisions.  This  is  the  majority-vote  strategy  which  possesses  the 
following  important  properties. 

1.  It  can  be  formulated  as  a sequential  procedure. 

2.  Its  n-sample  error  rate  can  be  evaluated  as  a function  of 
the  class-conditional  error  probabilities  of  the  simple 
decision  rule. 

3.  Its  error  rate  for  two  equally  likely  classes,  operating 
with  a decision  rule  of  specified  average  misclassification 
probability,  increases  monotonically  with  classification 
bias,  B,  of  the  decision  rule;  where  B is  a decision  rule 
parameter  defined  as  the  difference  between  class- 
conditional  error  probabilities  for  the  problem. 

The  paper  describes  the  application  of  these  results  in  a practical 
nonparametrlc  pattern  classification  problem  where  B is  shown  to  be 
a useful  criterion  for  decision  rule  or  feature  set  evaluation. 
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SAMPLE  FUNCTION  REGULARITY  OF  SHOT  PROCESSES,  Robert  Lugannanl  (De- 
partment of  Applied  Physics  and  Information  Science,  University  of 
California,  San  Diego,  La  Jolla,  California  92093).  This  paper  Is 
concerned  with  continuity  and  differentiability  of  the  sample  func- 
tions of  shot  processes.  A knowledge  of  these  properties  is  re- 
quired in  applications  in  which  it  is  desired  to  apply  classical 
analysis  techniques  to  the  individual  sample  functions  of  the 
process.  Sufficient  conditions  are  given  for  the  sample  functions 
to  be  continuous  and  to  be  continuously  differentiable  and  it  is 
shown  that  these  conditions  are  often  satisfied  in  practice.  The 
proof  is  based  on  a new  bound  for  the  probability  of  large  devia- 
tions of  the  process.  This  bound  has  considerable  independent 
interest  and  its  applicability  is  not  limited  to  the  study  of  sample 
function  properties.  With  the  exception  of  a few  special  cases, 
convenient  expressions  for  the  amplitude  distribution  of  a shot 
process  are  not  known,  and  in  these  instances  the  bound  provides 
valuable  information  concerning  its  behavior. 


SPACE-TIME  CODING  FOR  IMPROVED  OPTICAL  LINE-SCAN  IMAGING,  Stanley 
R.  Robinson  (School  of  Engineering,  Air  Force  Institute  of  Technology, 
Department  of  Electrical  Engineering,  Wright-Patterson  AFB,  Ohio 
45433).  An  interesting  form  of  active,  high  resolution  Imaging  is 
that  obtained  by  an  optical  line  scan  system.  In  such  a system  the 
use  of  a modulated  laser  would  make  possible  slant-range  (or  height) 
as  well  as  reflectivity  measurements  for  each  resolution  cell.  This 
paper  considers  the  problem  of  efficiently  using  multiple  solid  state 
laser  sources  to  improve  the  range/reflectivity  estimation  perfor- 
mance of  such  a system.  Only  direct  detection  receivers  and  power 
(intensity)  modulated  sources  are  considered  viable  candidates. 

We  demonstrate  that  multiple  sources  can  be  efficiently  used  if  each 
laser  is  focused  to  a spatial  resolut ion  cell  disjoint  from  all  other 
sources  and  if  each  is  modulated  with  a waveform  (or  code  word) 
which  is  essentially  temporally  orthogonal  from  all  others.  Thus 
each  spatial  resolution  cell  is  coded  with  a unique  temporally  modu- 
lated signal  which  can  be  decoded  by  the  receiver. 

The  detector  used  in  the  receiver  is  assumed  to  be  an  ideal  photon 
detector,  i.e.,  its  output  is  modeled  as  a conditional  Poisson  point 
process.  The  maximum  llklhood  (ML)  range/reflectance  estimator 
structures  are  presented;  an  indication  of  the  variance  of  the 
localized  estimate  is  obtained  through  the  Cramer-Rao  lower  bound. 

It  is  shown  that  both  variances  will  improve  by  as  much  as  where 

n is  the  number  of  sources.  Motivated  by  the  variance  of  the  range 
estimate,  we  select  an  ON-OFF  modulation  scheme  whereby  the  laser 
is  modulated  by  a Pseudonoise  (PN)  sequence  whose  period  is  chosen 
to  be  the  dwell  time.  We  conclude  that  acceptable  estimation  per- 
formance in  both  range  and  reflectance  in  each  cell  can  be  attained 
with  such  a multiple  source-modulation  format. 
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INTERFRAME  CODING  OF  VIDEOTELEPHONE  PICTURES  USING  MOVEMENT  COM- 
PENSATION - RECENT  SIMULATIONS,  B.G.  Haskell  and  D.K.  Shanna  (Bell 
Laboratories,  Holrodel,  New  Jersy  07733).  Recent  results  are  re- 
ported for  adaptive  interframe  coding  which  takes  advantage  of  the 
fact  that  moving  areas  in  one  television  frame  are  often  very 
similar  to  corresponding  areas  in  the  previous  frame  except  for  a 
linear  translation.  Minimum  M.S.E.  linear  prediction  and  interpola- 
tive  prediction  using  the  Limb  Velocity-Measure  were  employed  to 
code  30  successive  videotelephone  frames  containing  movement. 
Entropies  ranged  between  1 and  2 bits  per  moving-area  pel.  A video- 
tape of  coded  pictures  will  be  shown. 


NEAR-OPTIMAL  ESTIMATION-DETECTION  SCHEME  FOR  POISSON-DRIVEN 
PROCESSES,  S.  Au  and  A.H.  Haddad  (Department  of  Electrical  Engineer- 
ing and  Coordinated  Science  Laboratory,  University  of  Illinois, 

Urbana,  Illinois  61801).  A near-optimal  scheme  for  the  estimation 
of  signals  with  linear  models  and  a Poisson  process  input  is  proposed. 
The  scheme  is  nonlinear  and  is  based  on  the  detection  of  the  input 
jumps  in  small  subintervals  which  are  then  used  to  estimate  the  sig- 
nal. The  scheme  is  suitable  for  the  case  of  small  Incident  rate  X 
of  the  Poisson  input.  The  performance  is  compared  to  that  of  opti- 
mal linear  filters  which  are  optimal  for  such  processes  only  for 
large  X.  The  results  Indicate  that  there  exists  a rate  X*  such  that 
the  proposed  scheme  performs  better  in  the  MMSE  sense  than  the  linear 
scheme  for  X < X*. 


PERFORMANCE  FOR  BINARY  DECISION  PROBLEMS  WITH  POINT  PROCESS  OBSERVA- 
TIONS, J.L.  Hibey  (Dynamics  Research  Corporation,  Systems  Division, 

60  Concord  Street,  Wilmington,  MA  01887),  D.L.  Snyder  and  J.H.  van 
Schuppen  (Washington  University,  St.  Louis,  MO  63130).  We  are  con- 
cerned with  the  evaluation  of  error-probability  bounds  for  binary 
detection  problems  Involving  point  process  observations.  These 
bounds  are  of  interest  because  the  computation  of  the  exact  probabil- 
ities of  error  is  usually  mathematically  Intractable.  The  basic 
approach  we  employ  is  the  application  of  martingale  theory  to  detec- 
tion and  estimation  problems. 

The  decision  strategy  we  employ  is  the  generalized  likelihood-ratio 
test,  and  the  bounding  technique  is  due  to  Chernoff.  A measure 
transformation  technique  allows  us  to  obtain  an  expression  for  the 
Chernoff  bound  in  terms  of  an  expectation  of  a multiplicative  func- 
tional of  the  conditional  mean  signal  (rate  process)  estimates.  If 
the  processes  involved  are  Markovian,  we  are  then  able  to  represent 
the  above  expression  as  a solution  to  a backward  Kolmogorov  equation. 
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The  above  procedure  Is  essentially  repeated  when  the  optimal  esti- 
mates are  replaced  by  suboptimal  estimates.  These  results  are 
particularly  important  because  the  optimal  filtering  equations  can- 
not be  solved  exactly,  numerical  solutions  are  computationally 
Infeasible,  the  optimal  estimates  are  non-Markovlan , and  explicit 
systems  for  generating  the  exact  estimates  are  known  only  in  a few 
cases. 

We  then  Illustrate  our  procedure  with  examples  motivated  by  problems  of 
of  Interest  in  optical  communications. 


EXPERIMENTAL  PERFORMANCE  OF  POINT  PROCESS  ESTIMATORS  OF  OPTICAL 
PULSE  DELAY,  F.  Davidson  and  L.  Stephens  (Department  of  Electrical 
Engineering,  The  Johns  Hopkins  University,  Baltimore,  Maryland 
21218) . A series  of  experiments  was  performed  to  evaluate  the  per- 
formance of  minimum  mean  square  error  (MMSE)  estimates,  x',  of  the 
delay,  t,  of  Gaussian  and  rectangular  shaped  optical  pulses  from 
observed  photoelectron  emission  times.  In  the  absence  of  backgound 
radiation,  the  mean  square  error  for  rectangular  pulses  (13  usee 
wide,  rise  time  of  0.5  psec)  was  found  to  decrease  as  as  pre- 
dicted by  Bar-David,  where  Q is  the  optical  pulse  energy.  Gaussian 
shaped  pulses  of  equal  peak  intensity  and  identical  Q were  found  to 
have  the  predicted  Q~^  dependence  in  the  mean  square  error  of  esti- 
mates of  T. 

MMSE  estimates  in  the  presence  of  background  radiation  of  constant 
intensity  ig  were  found  from  numerical  solutions  to  equations  of 
evolution  for  the  posterior  conditional  density  of  x found  by 
Snyder.  At  equal  pulse  energies  and  peak  slgnal-to-nolse  ratios, 
Xs/Xq,  the  rectangular  pulses  were  found  to  give  superior  perfor- 
mance over  the  Gaussian  shaped  pulses  for  values  of  X^/Xq  in  excess 
of  10.  As  the  ratio  Xg/Xg  approached  unity,  the  mean  square  errors 
in  x'  for  the  two  differently  shaped  pulses  were  observed  to  become 
nearly  equal.  At  high  signal  to  noise  ratios  (l.e.,  > 10),  the 
mean  square  error  in  ^ for  the  Gaussian  shaped  pulse  was  found  to 
be  nearly  equal  to  the  lower  bound  found  by  Snyder. 


IMAGE  MODELING  WITH  APPLICATION  TO  MEASUREMENT,  J.W.  Burnett  (TRW 
Systems,  1 Space  Park,  Redondo  Beach,  CA  90278),  and  T.S.  Huang 
(School  of  Electrical  Engineering,  Purdue  University,  West  Lafayette, 
Indiana  47907).  We  develop  a fast  algorithm  for  pulse  width  esti- 
mation from  blurred  and  nonlinear  observation  in  the  presence  of 
signal-dependent  noise.  The  problem  is  motivated  by  the  need  for 
accurate  measurements  from  remotely  sensed  photographs.  Ideally, 
reflected  light  intensity  from  an  object  and  its  background  will  be 
approximately  piecewise  constant  with  discontinuities  at  the  edges. 
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However,  after  the  reflected  light  has  passed  through  the  imaging 
system,  the  edges  have  been  blurred.  Film  then  responds  nonlinear ly 
to  the  Incident  light  and  adds  signal-dependent  noise. 

The  problem  is  approached  by  modeling  the  edge  signal  (reflected 
light  intensity)  as  a discrete-position,  finite-state  Markov  process. 
Sample  functions  of  such  a process  are  graphically  represented  by  a 
path  through  a trellis.  Blurred  versions  of  these  signals  are 
similarly  represented.  By  assigning  a cost  to  each  branch  of  the 
trellis,  a maximum  a posteriori  probability  sequence  estimate  of  the 
signal  is  computed  by  finding  the  minimum  cost  path  through  the 
trellis.  The  Viterbi  algorithm  is  Introduced  as  an  efficient  means 
of  finding  the  minimum  cost  path  through  the  trellis. 

The  algorithm  is  applied  to  the  measurement  of  a road  in  an  aerial 
photo  taken  at  an  altitude  of  5000  feet.  The  resulting  width  esti- 
mate is  accurate  to  within  a few  Inches. 


ENCODING  OF  IMAGES  ON  A TWO-COMPONENT  SOURCE  MODEL,  Johnson  K.  Yan 
and  David  J.  Sakrlson  (Department  of  Electrical  Engineering  and 
Computer  Science  and  the  Electronics  Research  Laboratory,  University 
of  California,  Berkeley,  California  94720).  Setting  aside  the  issue 
of  complexity,  transform  codes  seem  to  perform  the  most  efficiently 
for  transmission  of  grey-scale  still  images.  For  large  raster  sizes,  a 
Fourier  transform  code  is  optimum  under  the  assumption  of  a Gaussian 
Source  and  weighted-square-error  distortion  criterion.  A weighted- 
square-error  distortion  criterion  on  log-intensity  does  not  repre- 
sent too  badly  the  criterion  of  a viewer's  visual  system  when  errors 
are  just  barely  visible.  However,  sample  Images  look  nothing  like 
sample  fields  from  a Gaussian  random  field.  A better  code  can  thus 
result  from  a better  modeling  of  the  source.  We  model  the  source  by 
breaking  a sample  field  up  Into  two  components:  a "discontinuous 
component"  representing  abrupt  changes  due  to  distinct  objects,  and 
a continuous  component  representing  shading  and  texture.  The  dis- 
continuous component  Is  represented  as  a ruled  surface,  horizontal 
scan  lines  across  It  consisting  of  straight  line  segments.  The 
breakpoints  of  these  segments  are  quantized  and  differentially 
encoded.  The  continuous  component  Is  the  remainder  after  subtract- 
ing the  discontinuous  component  from  the  original;  it  has  the 
behavior  of  a Gaussian  field  and  is  encoded  by  a transform  code. 
Simulations  reveal  considerable  Improvements  over  conventional 
transform  coding  of  the  whole  image. 
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RATE-DISTORTION  WITH  A FULLY  INFORMED  DECODER  AND  PARTIALLY  INFORMED 
ENCODER,  Toby  Berger  and  Ming  U.  Chang  (School  of  Electrical  Engi- 
neering, Cornell  University,  Ithaca,  N.Y.  14853).  Consider  the  fol- 
lowing generalization  of  the  rate-dlstorion  problem  of  Wyner  and 
Zlv.  The  discrete  memoryless  source  (dms)  {Xj^}  Is  to  be  encoded  for 
transmission  to  a destination  at  a rate  of  Rj^  bits  per  second  (bps). 
Side  information  is  available  at  said  destination  in  the  form  of 
another  dms,  (Yj^}.  The  joint  distribution  of  and  Y^.  which  does 
not  depend  on  k,  is  denoted  by  P.  The  generallzaton  is  that  infor- 
mation about  {Yj^}  may  be  supplied  to  the  encoder  at  a rate  of  R bps; 
Wyner  and  Zlv  treated  the  special  case  R«0. 

Let  Rx  (D,R)  denote  the  least  value  of  R^  that  sufficles  to  permit 
reconstruction  of  {X;^}  at  the  destination  with  fidelity  D as  assessed 
by  means  of  a memoryless  distortion  measure,  p(x,x).  By  proving  a 
coding  theorem  we  show  that  Rx(D,R)  ^ inf  I(X;Z|y,W),  where  (X,Y)  is 
distributed  according  to  P and  the  Inflmum  is  over  all  discrete  ran- 
dom variables  W and  Z that  satisfy  the  following  four  conditions: 


(1) 


(ii) 

I 

i (iii) 

(iv) 


X, Y,W  is  a Markov  chain 

Y,  (X,W) , Z is  a Markov  chain 
I(Y;W|X)  ^ R 

there  exists  a function  f such  that  X A f(Z,Y)  satisfies 
Ep(X,)b  i D. 


We  also  show  that  Rx(D,R)  i inf  I(X;Z1y,W)  if  the.  Inflmum  is  taken 
over  all  discrete  W and  Z that  need  satisfy  only  (1),  (ill),  and 
(Iv)  above.  Both  our  upper  and  our  lower  bound  are  non- trivial  in 
the  sense  that,  for  all  Rr (0,H(y|x)) , the  former  is  less  than  or 
equal  to  the  Wyner-Ziv  rate-distortion  function  and  the  latter  is 
greater  than  or  equal  to  the  conditional  rate-distortion  function. 
For  R “ 0 the  upper  bound  reduces  to  the  answer  provided  by  Wyner 
and  Zlv,  and  for  R 2 H(y1x)  the  upper  and  the  lower  bound  both  equal 
the  conditional  rate-distortion  function  R^^i  (D). 


MULTITERMINAL  SOURCE  CODING,  Toby  Berger  and  Sul-Yln  Tung  (School  of 
Electrical  Engineering,  Cornell  University,  Ithaca,  N.Y.  14853).  Let 
{1^  “ (Xlk»'-*^k)»  k ■ 0,  ± 1,  i 2,...)denote  a sequence  of  inde- 
pendent drawings  from  a known  n-varlate  distribution  Pjf.  The  sources 
{Xik,  k ■ 0,  ±1,  ±2,...},  1 s i s n,  can  be  encoded  aF  respective 
rates  Ri  for  transmission  to  a common  destination.  The  rate  vector 
R “ (Ri,...,Rn)  is  D “ (Di,...,!)^^)  - admissible  if,  on  the  basis  of 
all  n encoder  outputs  delivered  to  the  destination,  it  is  possible 
to  reconstruct  there  with  average  distprtlon  Dj^  or  less  as 

assessed  by  means  of  the  memorylass  distortion  measure  pj^(x,x), 

1 s 1 s n.  Let  R(D)  denote  the  region  of  all  ^admissible  rate 
vectors. 
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We  have  established  conditions  sufficient  to  ensure  that  Re  R(D) ; 
we  describe  them  here  only  for  the  case  of  n“2.  Let  X =■  (Xx,X2)  be 
distributed  according  to  Px*  Let  Rin(D)  denote  the  set  of  all  R » 
(R1,R2)  such  that  there  exTsts  a random  vector  Y = (Y^,Y2)  that 
satisfies 

(1)  I(Y^;(X2,Y2)|x^)  - I(Y2;(X^,Yp|X2)  * 0 

(li)  I(X^;YJY2)  ^ R^,  I(X2;Y2|Y^)  ^ I(X^;Yp  + 
I(X2;Y2|Yp  S Rj  + R2 

(lii)  there  exists  functions  f^  and  f2  such  that,  where 
we  have  E p^(X^,$^)  ^ D^,  1«1,2. 

Then  R(D) 3 Rin(D). 

We  also  show  that  R(D)Q  ^out(2.)>  where  Rout^®^  defined  in  the 
same  manner  as  is  Rin(£.)  except  that  (i)  and  (li)  are  replaced  by 

(i)’  l(Yj;X2|Xj^)  = I(Y2;XJX2)  = 0 

(li)’  I(X;YJY2)  ^Rj,  I(X;Y2|Yj)  s R2,  I(X;Y)  s R^  + R2 

In  general,  Rin(D)  is  a strict  subset  of  Rout ^2.^  • 

2 2 

For  a bivariate  Gaussian  Px  with  parameters  Op  02*  and  p,  let 

D(D)  - {(d^,d2):  d^  i i,je{l,2},  1 4 j}. 

Then  R e R^^  (D)  iff  there  exists  (dj^  ,d2) eD(D)  such  that,  where 

6 - 1 +yjl  + 4p^djd2(l-p^)"^  , 

"i  * '2  4 [23^] 

Rj  log  [ 2 ' ^2  1 ’ 

I (l-p^)Bd^  - 2p’^djd2J 

This  result  is  extended  to  jointly  stationary  Gaussian  random 
sequences. 
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CODING  RELATIVE  TO  FIDELITY  CRITERIA  FOR  CORRELATED  SOURCES  WITH 
MEMORY,  Jim  K,  Omura  (System  Science  Department,  University  of 
California,  Los  Angeles,  CA)  and  Kim  B.  Housewright  (Systems  Science 
Department,  University  of  California,  Los  Angeles,  CA  and  Hughes 
Aircraft  Company,  Fullerton,  CA) . Recent  research  efforts  have  pro- 
vided partial  characterizations  of  (l.e.,  "inner"  and  "outer"  bounds 
on)  the  rate-distortion  surfaces  of  multiterminal  systems  Involving 
memoryless  sources.  In  this  short  paper  we  describe  an  extension  of 
those  findings  to  systems  involving  sources  with  ergodic  memory 
structure.  We  obtain  an  "inner"  bound  which  is  cosmetically  similar 
to  the  result  for  the  memoryless  case,  but  is  defined  in  terms  of 
regions  associated  with  a constrained  set  of  stochastic  processes. 

In  fact,  our  bound  compares  to  the  previous  results  in  exactly  the 
same  manner  that  the  "process  definition"  of  the  rate  distortion 
function  compares  to  the  memoryless  result  for  a single  source- 
single  user  system.  We  have  also  developed  similar  bounds  for  more 
general  networks,  but  the  complexity  of  the  requisite  notation  makes 
them  unsuitable  for  discussion. 


CODE  CONSTRUCTION  FOR  THE  T-USER  BINARY  ADDER  CHANNEL,  S.C.  Chang 
(Electrical  Engineering  Department,  University  of  Hawaii,  Honolulu, 
HI  96822)  and  E.J.  Weldon,  Jr.  (Electrical  Engineering  Department, 
University  of  Hawaii,  Honolulu,  HI  96822,  and  Adtech,  Inc.,  P.O. 

Box  10415,  Honolulu,  HI  96816).  Coding  schemes  for  the  discrete 
memoryless  T-user  adder  channel  are  investigated  in  this  paper. 

First,  the  capacity  region  of  the  noiseless  T-user  adder  channel, 
and  the  maximal  achievable  rate  for  T-user  uniquely  decodable  codes 
are  derived.  Second,  code  constructions  are  presented,  including  a 
class  of  T-user  uniquely  decodable  codes  with  rates  asymptotically 
equal  to  the  maximal  achievable  value.  A decoding  algorithm  for 
these  codes  is  also  presented.  Finally,  a class  of  T-user  error- 
correcting  codes  for  the  noisy  adder  channel  is  constructed. 


ON  A CLASS  OF  6-DECODABLE  CODES  FOR  A MULTIPLE-ACCESS  CHANNEL,  Shu 
Lin  (Department  of  Electrical  Engineering,  University  of  Hawaii, 
Honolulu,  Hawaii  96822),  Tadao  Kasaml  (Faculty  of  Engineering  Science, 
Osaka  University,  Toyonaka,  Osaka  560,  Japan),  and  Saburo  Yamamura 
(Department  of  Measurement,  Kobe  University  of  Mercantile  Marine, 

Kobe,  Japan).  Consider  a multiple-access  communication  system  with 
two  users  which  employs  two  binary  block  codes  of  the  same  length 
n.  Cl  and  C2.  During  a message  interval,  each  user  chooses  a code 
word  from  his  code  and  both  users  transmit  simultaneously  over  a 
common  channel.  The  channel  combines  the  two  transmitted  code  words 
into  a single  vector  r over  a certain  alphabet.  A single  decoder  at 
the  receiving  end  processes  r and  decodes  it  in  two  code  words,  one 
in  Cl  and  the  other  in  C2.  The  code  pair  (Ci,C2)  used  in  the  above 
system  is  referred  to  as  a two-user  code. 
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In  this  paper,  we  Investigate  block  coding  for  a particular  memory- 
less two-input  single-output  multiple-access  channel.  First,  we 
present  a class  of  t-error-correcting  two-user  codes  which  we  call 
6-decodable  codes  where  t =■  [_(6-l)/2j.  Then,  we  derive  lower  bounds 
on  the  achievable  rates  of  codes  in  this  class  for  different  ranges 
of  t/n.  We  show  that  for  certain  range  of  t/n,  there  exist 
good  2-user  codes  (Ci,C2)  with  rates  (Rj,R2)  lying  above  the  time- 
sharing line,  l.e.,  ^^+^2  ^ 


DETERMINISTIC  CODES  FOR  SYNCHRONOUS  AND  ASYNCHRONOUS  COMMUNICATION 
OVER  THE  REAL  ADDER  MULTIPLE  ACCESS  CHANNEL,  Michael  A.  Deaett 
(General  Electric  Company,  Pittsfield,  Massachusetts)  and  Jack  Keil 
Wolf  (Department  of  Electrical  Engineering,  University  of  Massachu- 
setts, Amherst,  Massachusetts).  Deterministic  codes  have  been  found 
for  the  multiple  access  real  adder  channel  with  L users.  The 
cases  of  synchronized  and  of  nonsynchronized  Inputs  to  the 
channel  both  are  considered.  Both  variable  length  and  block  coding 
schemes  are  utilized.  The  rates  of  these  codes  are  compared  to  the 
maximum  achievable  rates  predicted  by  random  coding. 
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ON  MEAN- SQUARE  DECISION  FEEDBACK  EQUALIZATION  AND  TIMING  PHASE, 
Jack  Salz  (Bell  Telephone  Laboratories,  Holmdel,  NJ  07733).  We 
analyze  the  effect  of  timing  phase  on  the  performance  of  digital 
data  receivers  which  employ  decision  feedback  equalization.  It 
has  long  been  conjectured,  and  verified  by  computer  simulations, 
that  decision  feedback  equalizers  are  considerably  less  sensitive 
to  the  choice  of  timing  phase  than  are  conventional  transversal 
linear  equalizers.  We  develop  the  theoretical  machinery  which 
provides  a rationale  for  these  observations.  It  follows  from  our 
results  that  for  typical  operating  conditions  a 1-2  dB  penalty 
can  be  incurred  by  choosing  a bad  timing  phase  In  decision  feed- 
back, while  the  penalty  can  be  an  order  of  magnitude  greater  than 
this  (In  dB)  in  conventional  linear  equalizers. 


PERFORMANCE  OF  AN  ADAPTIVE  EQUALIZATION  TECHNIQUE  FOR  LINEAR  FM 
SIGNALS,  Laurence  B.  Mllsteln  (Department  of  Applied  Physics  and 
Information  Science,  University  of  California  at  San  Diego,  La 
Jolla,  California  92093).  Linear  FM  signals  have  a variety  of 
applications  in  communications  and  radar,  most  notably  in  pulse 
compression  radars.  However,  they  suffer  serious  distortion 
problems  when  used  over  channels  which  exhibit  large  phase  non- 
llnearltles.  This  is  because  the  channel  disperses  the  pulse  in 
time  over  possibly  many  times  its  original  width.  Consequently, 
the  need  for  equalizatio'  Is  clear,  and  adaptive  equalization  is 
desirable  If  one  assumes  the  channel  is  slowly  changing  with  time. 
In  this  paper,  a new  equalization  technique  designed  for  linear  FM 
signals  operating  over  channels  characterized  primarily  by  nonlin- 
ear phase  characteristics  will  be  described  and  analyzed  (l.e.,  it 
will  be  assumed  that  the  amount  of  phase  distortion  of  the  channel 
is  significantly  greater  than  the  amount  of  amplitude  distortion). 
The  equalizer  is  designed  to  approximate  as  closely  as  possible 
the  inverse  of  the  channel  so  that  the  combination  of  it  and  a 
filter  matched  to  the  original  transmitted  waveshape  will  yield  the 
desired  pulse  compression.  For  high  input  signal-to-noise  ratios, 
straightforward  implementation  of  the  above  equalizer  is  presented. 


APPLICATION  OF  FAST  KALMAN  ESTIMATION  TO  ADAPTIVE  EQUALIZATION,  D.D 
Falconer  (Bell  Telephone  Laboratories,  Holmdel,  NJ  07733)  and  L. 
Ljung  (Department  of  Electrical  Engineering,  Llnkoplng  University, 
S-581  83  Linkbplng,  Sweden).  Very  rapid  initial  convergence  of  the 
equalizer  tap  coefficients  is  a requirement  of  many  data  communica- 
tion systems  which  employ  adaptive  equalizers  to  minimize  inter- 
symbol  interference.  As  shown  in  recent  papers  by  Godard,  and  by 
Gltlin  and  Magee,  the  Kalman  estimation  algorithm  is  applicable  to 
the  estimation  of  the  optimal  (minimum  MSE)  set  of  tap  coefficients 
It  was  furthermore  shown  to  yield  much  faster  equalizer  convergence 
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than  chat  achieved  by  the  simple  estimated  gradient  algorithm, 
especially  for  severely  dlstored  channels.  We  show  how  certain 
"fast  Kalman  estimation"  techniques,  originally  Introduced  by  Morf 
and  Ljung,  and  with  complexity  proportional  to  the  number  of  equal- 
izer taps,  can  be  adapted  to  the  equalizer  adjustment  problem.  The 
fast  algorithm,  applicable  to  both  linear  and  decision  feedback 
equalizers,  exploits  a certain  shlf t-ln-varlance  property  of  succes- 
sive equalizer  contents.  It  achieves  the  same  fast  convergence  as 
the  traditionally- Implemented  Kalman  algorithm,  whose  complexity  Is 
proportional  to  the  square  of  the  number  of  taps.  The  rapid  con- 
vergence properties  of  the  fast  Kalman  adaptation  algorithm  are 
confirmed  by  simulation. 


REDUCED-COMPLEXITY  VITERBI  ALGORITHM  RECEIVER,  Ghassan  Kawas-Kaleh 
(Ecole  Natlonale  Superleure  des  Telecommunications,  75634  Paris 
CEDEX  13,  France).  A generalized  class  of  decoders,  which  contains 
both  decision-feedback  equalizer  and  maximum-likelihood  Viterbi 
algorithm  detector,  is  presented.  A reduced  complexity  equalizer 
can  be  obtained  by  making  the  Viterbi  algorithms  work  on  a small 
number  of  the  channel  impulse  response  samples;  the  intersymbol 
Interference  resulting  from  other  samples  is  estimated  using  tenta- 
tive decisions  available  in  the  algorithm  survivors. 

The  proposed  equalizer  has  the  advantage  of  effecting  a decrease  in 
residual  Intersymbol  interference  at  the  decoder  input,  when  Insert- 
ing a spectrum  shaping  prefilter,  without  increasing  complexity. 

Equalizer  performances  are  analyzed,  and  simulation  results  are 
presented.  They  show  a net  superiority  of  the  proposed  equalizer 
compared  to  others. 


A CLASS  OF  METHODS  FOR  COMPUTING  ADAPATIVE  EQUALIZER  PARAMETERS, 

P.  Cordes  (FG  Theorle  der  Slgnale,  Technische  Hochschule,  Darmstadt, 
F.R.,  Germany).  Transmitting  digital  data  over  bandllmited  channels 
at  high  speed  leads  to  distortions.  In  order  to  receive  the  data 
without  error  adaptive  equalizers  are  used  to  equalize  the  channel. 

Based  on  the  mean-square  error  criterion  methods  for  determining 
the  parameters  of  adaptive  equalizers  have  been  specified  and 
analyzed  recently. 

In  this  paper  it  Is  shown  how  most  of  these  procedures  can  be  com- 
bined to  a common  class.  Systematic  analysis  of  the  optimal 
algorithm  given  by  Kalman  leads  to  new  methods.  They  differ  in  com- 
puting effort  and  speed  of  convergence. 

Computer  simulations  Illustrates  the  results. 
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THE  EFFECTS  OF  LARGE  INTERFERENCE  ON  DIGITALLY  IMPLEMENTED  ADAPTIVE 
ECHO  CANCELLERS,  R.D.  Gltlln  and  S.B.  Weinstein  (Bell  Telephone 
Laboratories,  Holmdel,  NJ  07733).  Adaptive  mean-square-tapped-delay- 
line  echo  cancellers,  in  either  voice  or  data  applications,  are  con- 
ventionally designed  to  stop  adjustment  during  periods  of  "double- 
talking",  l.e.,  when  a large  information-bearing  signal  is  added  to 
the  echo  to  be  cancelled.  Continuous  adjustment  is,  however,  desir- 
able in  full-duplex,  two-wire  data  transmission.  We  address  the 
problem  of  tap  adjustment,  via  the  estimated-gradient  algorithm,  in 
the  possible  presence  of  a "double- talking"  signal. 

For  an  idealized  double-talking  model,  it  is  demonstrated  that  the 
memoryless  mciximum-likelihood  estimate  of  the  residual  echo  is  a linear 
function  of  the  canceller  output  signal  up  to  a certain  threshold, 
beyond  which  the  echo  estimate  falls  back  within  a hysteresis  region 
before  evolving  into  another  linear  function  with  much  smaller  slope. 
This  is  nearly  equivalent  to  the  strategy  of  abruptly  reducing  the 
step  size  of  the  adjustment  algorithm  when  double-talking  begins, 
and  is,  in  fact,  an  automatic  mechanism  for  recognizing  double-talking. 
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ON  DIGITAL  SIGNATURES  AND  PUBLIC-KEY  CRYPTOSYSTEMS,  R.L.  Rives t,  A. 
Shamir  and  L.  Adleman  (MIT  Laboratory  for  Computer  Science).  The 
operation  of  raising  a number  to  a fixed  power  modulo  a composite 
modulus  is  shown  to  be  sufficient  to  implement  "digital  signatures": 
a way  of  creating  for  a (digitized)  document  a recognizable,  un- 
forgeable,  document-dependent,  digitized  signature  whose  authenticity 
the  signer  can  not  later  deny.  This  scheme  has  obvious  applications 
in  the  design  of  "electronic  funds  transfer"  systems  or  "electronic 
mall"  systems,  since  here  the  messages  must  be  digitized  in  order  to 
be  transmitted.  Our  approach  is  to  provide  an  implementation  of  a 
"public-key  crytosystem" , an  elegant  concept  Invented  by  Diffle  and 
Heilman.  Such  a system  also  enables  enciphered  communication  between 
arbitrary  pairs  of  people,  without  the  necessity  of  agreeing  on  an 
enciphering  key  beforehand. 


HIDING  INFORMATION  AND  RECEIPTS  IN  TRAP  DOOR  KNAPSACKS,  Ralph  C. 

Merkle  and  Martin  E.  Heilman  (Department  of  Electrical  Engineering, 
Stanford  University).  Given  a vector  or  integers  a and  a number  S 
such  that  S«£-x,  where  x in  a binary  vector,  the  knapsack  problem  is 
to  find  X.  Because  this  is  an  NP-complete  problem  it  is  strongly  be- 
lieved to  be  difficult  to  solve  in  general.  A trap  door  knapsack  vector 
a makes  the  problem  appear  very  difficult  unless  one  possesses  trap 
door  Information  used  in  the  design  of  a.  Because  only  the  designer 
can  find  the  solution,  others  can  send  him  information  x hidden  in 
the  sum  S“a^»2C.  This  appraoch  differs  from  usual  cryptographic  systems 
in  that  a secret  key  need  not  be  exchanged. 

Methods  for  generating  and  solving  trap  door  knapsacks  will  be  de- 
scribed. When  implemented  in  circuitry  these  techniques  allow  anyone, 
regardless  of  mathematical  ability,  to  automatically  generate  a trap 
door  knapsack  vector  and  to  recover  information  sent  to  him  in  hidden 
form.  A skilled  opponent  cannot  discover  this  information  even  though 
he  knows  the  general  method  used  for  generating  the  trap  door  vector 

It  is  also  demonstrated  that  these  techniques  allow  the  generation  of 
receipts  (unf orgeable,  message  dependent,  digital  signatures). 

Receipts  are  more  valuable  than  usual  digital  authenticators  because 
they  allow  the  resolution  of  disputes  between  the  transmitter  and  the 
receiver  as  to  whether  a particular  message  was  sent. 


AN  IMPROVED  ALGORITHM  FOR  COMPUTING  LOGARITHMS  OVER  GF(p"')  AND  ITS 
CRYPTOGRAPHIC  SIGNIFICANCE,  Stephen  C.  Pohllg  and  Martin  E.  Heilman 
(Department  of  Electrical  Engineering,  Stanford  University,  Stanford, 
CA  94305).  An  improved  algorithm  is  presented  for  computing 
logarithms  over  GF(p),  where  p is  a prime  number.  Previously  pub- 
lished algorithms  require  0(p^'2)  complexity  in  both  time  and  space. 
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i 1 

The  algorithm  presented  here  requires  0(1  n "p  ‘Log  p ) opera- 

^11  1 

tlons,  0(Z  p "Log p ) bits  of  memory,  and  0(1  p 'Log  p ) pre- 
i 1 1 ^ X 1 

"l  \ 

computation  operations,  where  0 S r^^^  S 1,  and  p - '’’Pk 

the  prime  factorization  of  p - 1. 

A cryptographic  system  and  a user  authentication  system  based  on 
exponentiation  mod  p are  described.  The  above  mentioned  algorithm 
dictates  that  p - 1 should  be  chosen  to  have  a large  prime  factor 
so  that  Logatitlims  over  GF(p)  are  difficult  to  compute. 


This  improved  algorithm  directly  extends  to  all  finite  fields  GFCp”) 
Of  practical  interest  is  the  case  GF(2™)  when  2™-l  is  prime  (2™-l  is 
a Mersenne  prime).  Exponentiation  is  not  difficult  to  implement  for 
this  case.  Many  large  Mersenne  primes  are  known. 


A HIGH-GRADE  DATA  ENCRYPTION  ALGORITHM,  Bernard  J.  Pankowski  (Com- 
puter Sciences  Corporation,  6565  Arlington  Boulevard,  Falls  Church, 
Virginia  22046) , A high-grade  algorithm  for  encrypting  data  for 
transmission  or  storage  is  described.  It  develops  more  key  space 
than  the  NBS  algorithm,  and  offers  an  efficient  software  alternative 
to  hardware  encryption  for  applications  where  the  volume  of  data  re- 
quiring protection  does  not  justify  the  cost  of  acquiring  and  main- 
taining dedicated  encryption  hardware. 

Two  algorithm  variations  have  been  coded,  one  processing  only  print- 
able EBCDIC  characters,  and  one  scrambling  all  data  forms  (characters, 
integers,  and  floating-point  numbers).  Both  versions  produce  "com- 
putationally secure"  key  stream  ciphers  by  linking  two  congruence- 
type  random  generators  to  form  a single  long  sequence  (62  key  bits), 
which  combined  with  cipher  feedback  (6  or  8 key  bits),  produces  a 
cryptographic  key  stream  that  never  repeats. 

Under  control  of  this  key  stream,  a variation  of  the  Vlgenere  square 
(in  use  since  1586)  performs  either  encryption  or  decryption  using  a 
polyalphabetlc  substitution  on  characters  (or  bytes).  The  Vlgenere 
square  variant  is  equivalent  to  an  enciphered  (simple  substitution) 
modulo-n  half-adder  whose  permutations,  64 1 for  the  character  ver- 
sion and  2561  for  the  all  data  version,  add  378  and  2040  effective 
key  bits  respectively  to  the  two  algorithm  variations. 

No  method  of  "breaking"  the  algorithm  is  known  short  of  exhausting 
the  key  space,  an  economically  prohibitive  undertaking. 
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PARALLEL  PROCESSABLE  CRYPTOGRAPHIC  METHODS  WITH  UNBOUNDED  PRACTICAL 
SECURITY,  Jerome  Rothsteln  (Department  of  Computer  and  Information 
Science,  The  Ohio  State  University,  Columbus,  Ohio  43210).  Crypto- 
graphic methods  are  described  which,  at  reasonable  cost  to  users. 
Impose  prohibitive  work  penalties  on  code-breakers.  The  class  of 
cryptographic  transofrmatlons  Is  so  vast  that  codes  can  be  used 
once  and  discarded,  both  In  communication  and  In  data  security  ap- 
plications. We  use  the  groupold  string  formllsm  (earlier  shown  to 
combine  Turing  universality  with  parallel  computing  capability) 
specialized  to  quasigroups  for  unique  decodablllty . Coding  for 
error  detection  and  correction  can  be  done  compatibly  and  Independ- 
ently. It  appears  likely  that  crypto  and  reliability  encoding  can 
be  systematically  combined  to  decoy  code-breakers  Into  semantlclzlng 
their  decoding  of  reliability  procedures  Into  message  opaclflers. 
Identification,  authorization,  and  handshaking  procedures  can  be 
Integrated  Into  the  cryptographic  method  (with  total  update  capabil- 
ity) at  Incremental  user  cost  comparable  to  separate  cost  of  those 
procedures  or  less. 


AN  ALGORITHM  FOR  SOLVING  SIMPLE  SUBSTITUTION  CRYPTOGRAMS,  Lallt  Bahl, 
IBM  Thomas  J.  Watson  Research  Center,  P.O.  Box  218,  Yorktown  Heights, 
NY  10598).  A1  algorithm  for  automatic  decipherment  of  simple  sub- 
stitution ciphers  Is  presented.  Let  X denote  the  cleartext  message, 
and  Y the  encrypted  message.  X Is  treated  as  the  output  of  a Markov 
source,  whose  parameters  are  estimated  from  letter  n-gram  frequency 
counts  obtained  from  a large  sample  of  English  text.  The  encryption 
process  X-*Y  is  modeled  as  transmission  through  a lossless  channel 
with  unknown  transition  probabilities  p(.|.).  The  transition  matrix 
of  this  channel  Is,  of  course,  a permutation  matrix  containing  a 
single  1 In  each  row  and  column.  Our  aim  is  to  determine  p(.|.), 
given  Y and  the  source  statistics. 

Initially,  the  permutation  nature  of  the  channel  transition  matrix 
Is  Ignored,  and  Initial  estimates  of  p(.|.)  are  obtained  from  first 
order  letter  counts  of  Y.  The  Forwards  Backwards  algorithm  is  used 
to  iteratively  refine  these  estimates.  The  algorithm  adjusts  p(.|.) 
to  Increase  PrlY)  (which  is  a function  of  p(.|.)  and  the  source 
statistics).  As  entries  In  the  channel  transition  matrix  come  close 
to  1,  they  are  set  to  1.  The  process  stops  when  the  transition 
matrix  becomes  a permutation  matrix.  This  permutation  Is  the  key 
used  for  encrypting  and  Its  Inverse  can  be  applied  to  Y to  obtain  X. 

Using  a letter  2-gram  model  of  English,  this  algorithm  conlstently 
yielded  the  correct  answer  If  the  cryptogram  was  about  300  letters 
or  longer.  This,  of  '.ourse.  Is  considerably  longer  than  the  30  or 
so  letters  generally  needed  by  humans  to  break  simple  substitution 
ciphers.  Experiments  with  higher  order  n-gram  Markov  models  are 
planned . 
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ENCRYPTION  WITH  RANDOMLY  CHOSEN  UNKNOWN  FUNCTIONS,  Ingemar  Ingemarsson 
(Linkoplng  University,  Sweden).  An  encryption  unit  may  be  regarded 

M 

as  an  unknown  function.  That  is  a set  of  functions  {f^(x)}^^j^  from 

a finite  set  X onto  a finite  set  Y.  The  set  is  known  to  an  out- 
side observer.  The  key,  i,  and  thus  the  actual  function  fi^(x),  how- 
ever, is  known  solely  by  the  legitimate  user.  It  is  assumed  that  the 
outside  observer  can  test  the  encryption  unit  by  applying  r different 
inputs  xi,...,xr  and  observing  the  corresponding  outputs  yi,...,yr- 
Alternatively  the  inputs  Xj|^,...,Xj.  are  chosen  randomly.  Shannon 
studied  the  situation  where  the  outside  observer  did  not  know  the 
input  ("the  clear  text")  xi,...,Xj..  In  many  cases  it  is  more  realis- 
tic to  assume  knowledge  of  a limited  set  of  Inputs.  The  question  is 
to  what  extent  this  facilitates  the  cryptanalysis,  i.e.,  the  deriva- 
tion of  X from  y.  The  key,  1,  is  regarded  as  a random  variable. 

We  may  then  calculate  the  conditional  probabilities  p[y|x,R]  where  R 
denotes  the  set  of  observed  Inputs  and  outputs  X]^,...,Xj.  and  yj^,..., 
yj..  The  ideal  choice  of  unknown  function  from  the  designer’s  point 
of  view,  seems  to  be  one  where  the  output  y is  independent  of  R. 

Such  unknown  functions  were  called  unknown  functions  with  maximal 
uncertainty  and  analyzed  by  this  author. 

The  aim  of  this  paper  is  to  study  unknown  functions  where  the  M func- 
tions fi(x)  are  chosen  randomly  from  the  set  of  all  possible  functions 
from  the  finite  set  X onto  a finite  set  Y.  The  main  result  is  that 
there  is  a remarkable  and  very  fast  convergence  with  increasing  M in 
that  a vast  majority  of  all  unknown  functions  belong  to  a class  with 
nearly  the  same  conditional  probabilities  p[y|x,R].  Thus  even  if  it 
were  shown  that  there  exist  unknown  functions  with  maximal  uncertainty, 
nearly  all  unknown  functions  have  less  than  maxlmam  uncertainty.  For 
small  r,  however,  the  uncertainty  is  close  to  maximal.  (Uncertainty 
is  here  measured  as  the  conditional  entropy.)  Randomization  is  not 
only  a mathematical  tool.  It  can  also  be  used  to  produce  unknown 
functions,  to  be  used.  Tor  example,  in  Identity  control  units.  The 
convergence  asserts  that  almost  all  units  produced  in  such  a way 
have  similar  statistical  properties. 

When  unknown  functions  are  used  for  encryption  we  must  require  that 
the  functions  f£(x)  are  invertible.  Hence  the  analysis  described 
above  is  extended  to  invertible  unknown  functions,  i.e.,  unknown 
functions  where: 

f,  (x  ) ^ f,  (x  ) for  all  i and  x ^ x 
i m i n m n 

Finally  results  are  also  given  for  unknown  functions  which  are  ran- 
domly chosen  without  replacement  from  the  set  of  all  possible  func- 

M 

tlons.  This  guarantees  that  all  M functions  in  the  set  If^(x)}^_j^, 
are  distinct. 
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BOUNDS  ON  SECRECY  SYSTEMS,  Shyue-Chlng  Lu  (Telecommunication  Labora- 
tories, Ministry  of  Communications,  R.O.C.)-  The  problem  of  enciph- 
ering the  output  of  a discrete  message  source  is  considered.  The 
key  used  in  enciphering  is  transmitted  safely  to  the  user;  however, 
the  enciphered  message  is  Intercepted  by  an  "enemy". 

The  class  of  additive-like  instantaneous  block  (ALIB)  encipherers 
is  studies.  An  ALIB  encipherer  is  specified  by  an  addltlve-llke 
combiner  and  a set  of  key  words.  An  ALIB  encipher  operates  as  fol- 
lows: the  selected  key  determines  a particular  key  word  and  the 
cryptogram  is  produced  by  the  combiner  letter  by  letter  from  the 
message  and  the  particular  key  word.  The  strength  of  an  encipherer 
is  measured  by  the  probability  of  correct  decryptment,  the  probabil- 
ity that  the  non-zero  per  letter  Hamming  distortion  is  less  than  the 
tolerable  distortion,  the  number  of  keys  and  the  tolerable  distortion. 
It  is  shown  that  there  exists  an  ALIB  encipherer  with  key  rate  r, 
tolerable  distortion  A such  that  both  the  probability  of  correct 
decryptment,  p^,,  and  the  probability  that  the  non-zero  per  letter 
Hamming  distortion  is  less  than  the  tolerable  distortion;  p(A),  can 
be  made  arbitrarily  small  provided  the  key  rate  is  greater  than  a 
bound  on  the  cipher-distortion  function. 

The  concept  of  sphere  packing  is  used  to  derive  both  upper  and 
lower  bounds  on  p^  for  binary  additive  Instantaneous  block  encipher- 
ers and  memoryless  binary  sources.  In  this  case  p(A)  is  required  to 
be  zero  for  a prescribed  tolerable  per  letter  Hamming  distortion  A. 

For  ALIB  encipherers  and  discrete  message  sources  the  concepts  of 
random  ciphering  and  exponential  bounding  are  employed  to  obtain 

bounds  on  p and  p(A). 
w 

In  contrast  to  Shannon"s  "random"  cipher  result,  it  is  shown  that 
good  encipherers  exist  with  key  rates  less  than  the  message  redundancy. 
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ON  A NEW  CLASS  OF  OPTIMAL  DETECTORS:  THE  APPROXIMATE  IMPULSE 
DETECTION,  Michael  Negin  and  C.  Britton  Rorabaugh,  Jr.  (Department 
of  Electrical  Engineering,  Drexel  University,  32nd  and  Chestnut 
Streets,  Philadelphia,  PA  19104).  Matched  filters  are  often  used 
for  detecting  a signal  of  knovm  shape,  but  with  a random  time  of 
arrival.  In  the  case  where  multiple  signals  are  to  be  detected, 
the  matched  filter  is  useful  as  long  as  the  multiple  signals  do  not 
overlap  or  superimpose.  Since  the  matched  filter  la  optimized  with 
respect  to  an  amplitude  criterion  (peak  output  signal  to  root  mean 
squared  output  noise) , It  is  no  surprise  that  the  matched  filter 
can  perform  poorly  when  signals  overlap  in  time.  Several  investi- 
gators have  proposed  modified  Inverse  filters  to  solve  the  problem 
of  overlapped  signals.  Since  in  a noise-free  environment.  Inverse 
filters  produce  impulses  for  every  occurrence  of  the  signal  to  be 
detected,  it  seems  reasonable  to  try  to  use  them  as  detectors.  Un- 
fortunately in  the  noisy  environment.  Inverse  filters  usually  amplify 
noise  considerably.  The  modified  inverse  filters  discussed  in 
literature  have  typically  not  been  general  in  development  and  further- 
more usually  require  information  such  as  a desired  resolution  time. 

The  approach  taken  in  this  paper  is  one  that  formulates  an  optimal 
detection  criterion  that  combines  amplitude  and  resolution  criteria. 

The  criterion  essentially  results  in  a linear  filter  that  (in  the 
mean  square  sense)  optimally  estimates  an  impulse  train  that  corre- 
sponds to  the  time  of  arrival  statistics  of  the  signal  train  to  be 
detected.  Multiple  and  overlapping  signals  are  optimally  detected 
since  the  time-of-arrival  statistics  are  used  in  deriving  the  optimal 
filter. 

The  Approximate  Impulse  Detector  (AID)  is  given  by. 
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where  Fq((»))  is  the  optimal  filter,  S*(a))  is  the  conjugate  of  the 
signal  to  be  detected,  4nn(***)  the  power  spectral  density  of  the 
noise,  and  4ss(<J^)  “ S(u))  S*(u)).  4pp(w)  is  the  power  spectral  density 
of  an  Impulse  train  that  corresponds  to  the  time  of  arrival  statis- 
tics of  the  signal  train  to  be  detected. 

The  optimal  filter  has  been  simulated  in  a variety  of  signal  and 
noise  conditions  and  Indeed  has  considerably  better  resolution 
properties  than  the  matched  filter.  The  signal  to  noise  ratio  per- 
formance of  the  AID  filter  is  comparable  to  the  matched  filter. 
Synthesis  of  AID  filters  is  only  slightly  more  difficult  than  matched 
filter  synthesis.  Furthermore  it  can  be  seen  from  the  equation  that  in 
certain  high  noise  environments,  (4nn'^)  reduce  to  the 
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matched  filter.  In  low  noise  environments  (41  , AID  yields  an  in- 

verse filter.  In  moderate  noise  environments,  the  AID  filter  specifies 
an  optimal  detection  filter  for  a given  noise  and  signal  environment. 


SEQUENTIAL  PARTITION  DETECTORS  WITH  DEPENDENT  SAMPLING,  Roger  F. 

Dwyer  (Naval  Underwater  Systems  Center,  New  London,  CT  06320)  and 
Ludwlk  Kurz  (Polytechnic  Institute  of  New  York,  Department  of  Engi- 
neering and  Electrophysics,  Brooklyn,  NY  11201).  In  this  paper, 
the  theory  of  sequential  partition  detectors  with  dependent  sampling 
is  introduced.  A new  formulation  is  given  which  predicts  the  thresh- 
olds under  q-dependent  sampling  in  order  to  maintain  the  same  error 
probabilities  as  in  the  Independent  sampling  case.  A comparison  is 
made  between  independent  and  dependent  sequential  partition  detectors 
based  on  the  average  time  to  detection.  Under  stated  conditions  de- 
pendent sequential  partition  detectors  show  Improved  efficiency  for 
both  Lelunann  and  shift  of  the  mean  alternatives. 


A NEW  APPROACH  TO  ADAPTIVE  DETECTION,  B.  Picinbono  (Laboratoire  des 
Slgnaux  et  Systemes,  ESE  - Plateau  du  Moulon,  91190,  GIF,  France). 

The  statistical  detection  theory  of  signal  in  noise  is  generally 
presented  in  a non  adaptive  context.  In  such  a context  is  is  as- 
sumed that  some  statistical  properties  of  the  signals  and  the  noise 
are  known  beforehand.  For  example  the  likelihood  optimal  receiver 
needs  the  knowledge  of  the  probability  distributions.  Unfortunately 
it  is  often  difficult  to  have  such  a knowledge,  and  moreover  there 
are  in  general  Important  variations  of  the  statistical  parameters 
which  make  an  adaptive  processing  necessary. 

It  is  difficult  to  present  a general  theory  of  adaptive  and  optimal 
detection.  The  most. common  way,  but  not  necessarily  optimal,  is  to 
replace  in  the  structure  of  optimal  receivers  the  unknown  parameter 
by  a convenient  estimation.  But  to  perform  this  estimation  we  need 
a sample  of  the  observation  which  is  composed  of  only  noise.  Another 
way  is  to  extract  from  the  observation  a function  which  la  a Noise 
Alone  Reference  (N  A R)  and  which  can  give  informations  on  the  noise, 
even  if  a signal  is  present. 

In  this  paper  we  Introduct  the  notion  of  N A R and  we  give  some 
examples  appearing  in  detection  problems.  Moreover  we  show  that  the 
notion  of  N A R appears  very  naturally  in  the  physical  interpretation 
of  matched  filter  in  colored  noise.  Finally  we  give  some  indications 
of  the  use  of  such  a N A R in  detection  of  deterministic  signals  in 
noise  with  fluctuating  power. 
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STOCHASTIC  APPROXIMATION  IN  DETECTION,  S.C.  Lee  and  L.W.  Nolte  (De- 
partment of  Electrical  Engineering,  Duke  University,  Durham,  NC 
27706).  The  role  of  stochastic  approximation  in  detection  theory 
is  investigated  for  the  Gauss-Gauss  array  processing  problem.  The 
adaptive  least  mean-square  (LMS)  algorithm  is  used  as  a stochastic 
approximation  technique  for  both  detection  and  estimation.  In  the 
Gauss-Gauss  problem,  it  is  well  known  that  the  optimum  detector  can 
be  put  into  the  fo’^  of  an  estimator-correlator  in  which  the  mini- 
mum mean-square  error  estimator  (MMSE)  is  correlated  with  the  data. 

A suboptimum  detector  is  considered  here  in  which  the  adaptive  LMS 
algorithm  is  used  as  the  estimator  in  an  estimator-correlator 
structure. 

Detection  performance  is  analyzed  using  the  detectability  index 
both  for  optimum  and  suboptimum  detectors.  The  estimation  perform- 
ance of  the  MMSE,  MLE,  and  adaptive  LMS  estimators  are  also  com- 
puted using  the  mean-square  error.  It  is  shown  that  the  detectability 
of  the  suboptimum  detector  suffers  seriously  compared  to  the  detect- 
ability of  the  optimum  detector.  The  mean-square  error  of  the 
adaptive  LMS  estimator,  however,  converged  to  that  of  the  optimum 
estimator  (MMSE)  within  a reasonable  number  of  iterations  under  the 
signal-present  hypothesis. 


ROBUST  SEQUENTIAL  DETECTION  OF  SIGNALS  IN  NOISE,  A.H.  El-Sawy  and 
V.D.  VandeLinde  (Department  of  Electrical  Engineering,  The  John 
Hopkins  University,  Baltimore,  Maryland  21218).  The  problem  of 
sequential  detection  of  weak  signals  in  additive  noise  is  solved 
under  the  assumption  that  the  unknown  noise  density  function  is  a 
member  of  some  known  class  of  symmetric  densities.  Two  general 
approaches  to  the  design  of  receivers  which  as  asymptotically  most 
robust  in  a minimax  sense  are  established.  Both  receivers  guarantee 
an  upper  bound  on  the  probability  of  errors  of  the  first  and  second 
type  and  minimize  the  maximum  expected  risk  in  Bayes  sense.  The 
main  difference  between  them  is  that  the  first,  which  we  call  the 
M-sequential  detector,  requires  a growing  memory,  and  the  second, 
which  is  called  the  stochastic  approximation  sequential  detector, 
does  not.  A comparison  between  the  two  methods  is  provided  for  the 
special  case  when  the  noise  density  family  is  defined  by 
a 

F » (f:/  f(x)  dx  “ p,  f symmetric  and  continuous  at  ± a) 

“3 

In  addition  the  savings  in  the  expected  sample  size  over  some  non- 
sequential methods  have  been  calculated. 
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SIGNAL  DETECTION  THROUGH  NOISE  REMOVAL,  A.J.  Claus,  T.T.  Kadota, 
and  D.M.  Remain  (Bell  Telephone  Laboratories,  Inc.,  Murray  Hill  and 
Whippany,  NJ).  This  paper  deals  with  the  construction  of  an  ortho- 
normal set  of  vectors  which  is  useful  for  adaptive  side  lobe  manipu- 
lation of  array  beam  patterns.  The  vectors  allow  side  lobe  suppres- 
sion across  any  spatial  sector  and  they  depend  only  on  the  array 
geometry.  The  latter  feature  permits  the  numerical  values  of  vector 
components  to  be  pre-computed  and  permanently  stored  making  their 
use  attractive  for  real-time  adaptive  array  processing.  The  center 
of  the  sector  across  which  side  lobes  must  be  altered  determines 
array  sensor  steering  delays,  and  the  sector  width  specifies  the 
number  of  vectors  to  be  used. 

This  number  is  closely  related  to  the  essential  rank  of  some  fre- 
quency dependent  matrix.  The  matrix  is  subjected  to  a sequence  of 
linear  mappings,  each  mapping  producing  a vector  of  the  orthonormal 
set.  The  mappings  are  carried  out  at  frequencies  determined  by  pre- 
assigned error  bounds.  This  procedure  results  in  frequency  inde- 
pendent vectors  capable  of  side  lobe  manipulation  to  within  any 
degree  of  accuracy. 


} BOUNDS  ON  PROBABILITY  OF  ERROR  AND  SUBOPTIMAL  DETECTORS,  Alain  Fogel 

and  Stuart  C.  Schwartz  (Department  of  Electrical  Engineering  and 
Computer  Science,  Princeton  University,  Princeton,  NJ  08540).  A 
wide  class  of  optimum  likelihood  ratio  detectors  are  specified  in 
terms  of  a conditional  mean  estimate  (CME) , and  the  resulting  im- 
plementation is  an  estimator-correlator.  When  the  CME  is  either 
unknown  or  difficult  to  evaluate,  other  estimates  can  be  substituted, 
resulting  in  a class  of  sub-optimum  detectors.  Although  probabilities 
of  error  are  the  central  measure  of  performance  of  a detector,  these 
are  usually  not  available  and  bounding  techniques  such  as  the  Chernoff 
bounds  have  provided  an  alternative  approach  to  obtain  a tractable 
criterion  of  performance.  Using  a variational  approach,  we  investi- 
gate the  relationship  between  the  two  resulting  Chernoff  bounds  on 
the  probabilities  of  error  for  the  above  class  of  optimum  and  sub- 
optimum detectors.  Problems  considered  include:  continuous- time 
detection  of  a signal  in  white  Gaussian  noise;  signals  modulating  the 
rate  of  a doubly  stochastic  Poisson  process;  the  discrete-time 
estimator  correlator  known  to  be  optimum  when  the  noise  distribution 
belongs  to  the  exponential  family. 
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SIGNAL  FILTERING  WITH  GAUSSIAN  QUANTUM  STATES  AND  CANONICAL  MEASURE- 
MENT, R.O.  Harger  and  J.S.  Baras  (Electrical  Engineering  Department, 
University  of  Maryland,  College  Park,  Maryland  20742).  The  recently 
initiated  extension  of  classical  signal  filtering  accounting  for 
quantum  mechanical  limitations  on  measurement  is  here  continued  In- 
corportlng  measurement  effect  on  a quantion  state  which  depends  on  a 
vector  random  "signal"  process  and  evolves  according  to  the  Shrodlnger 
equation.  The  quantum  measurements  are  restricted  to  be  "canonical" 
and  a system  Hamiltonian  that  results  in  a Gauss-Markov  measurement 
outcome  sequence  is  assumed.  The  weighted  mean-square  error  is 
minimized  over  choice  of  current  quantum  measurement  and  linear 
weightings  of  past  measurements.  It  is  shown  that  the  optimal  weight 
matrices  satisfy  certain  normal  equations  and  necessary  and  sufficient 
conditions  for  the  optimal  canonical  measurement  are  given.  Suffi- 
cient conditions  for  the  "separation"  into  a canonical  measurement 
independent  of  past  measurements  and  classical  "post"  filtering  are 
given.  An  optical  communication  application  is  given. 


QUANTUM  STATE  PROPAGATION  AND  OPTICAL  CHANNELS,  Horace  P.  Yuen  (Re- 
search Laboratory  of  Electronics,  Cambridge,  Massachusetts  02139) 
and  Jeffrey  H.  Shapiro  (Research  Laboratory  of  Electronics  and  De- 
partment of  Electrical  Engineering  and  Computer  Science,  M.I.T., 
Cambridge,  Massachusetts  02139).  To  determine  the  ultimate  quantum 
limitations  on  the  performance  of  optical  communication  systems,  it 
is  essential  to  consider  optimum  quantum  state  generation  at  the 
transmitter  and  the  effect  of  the  propagation  channel  on  the  re- 
ceived state.  The  propagation  of  arbitrary  quantum  states  in  free 
space  is  developed  herein  as  a quantum  diffraction  theory.  This 
theory  yields  the  correct  classical  Huygens-Fresnel  limit,  and  justi- 
fies the  usual  semiclasslcal  propagation  treatment  of  coherent  state 
fields.  For  states  that  cannot  be  described  by  a well-behaved 
P-representation,  such  as  the  recently  discussed  two-photon  coherent 
states,  quantum  diffraction  theory  permits  separation  of  the  state- 
generation  problem  from  the  state-propagation  problem. 

This  allows  the  quantum  transmitter,  the  channel,  and  the  receiver 
to  be  represented  in  a convenient  framework  similar  to  the  semi- 
classical  theory  of  optical  communication.  This  framework  will  be 
used  to  examine  near-field  and  far-fleld  performance  potential  of 
two-photon  coherent  states.  Application  of  the  same  approach  to 
some  other  optical  channels  will  be  Indicated. 
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STRUCTURED  OPTICAL  DETECTION  OF  TWO-PHOTON  RADIATION,  Jesus  A. 

Machado  Mata,  Jeffrey  H.  Shapiro  and  Horace  P.  Yuen  (Department  of 
Electrical  Engineering  and  Computer  Science  and  Research  Laboratory 
of  Electronics,  Massachusetts  Institute  of  Technology,  Cambridge, 
M.iissachu8etts  02139).  Recent  theoretical  work  has  shown  that  novel 
quantum  states,  called  two-photon  coherent  states  (TCS) , have  sig- 
nificant potential  for  improving  free-space  optical  communications. 
Because  TCS  radiation  does  not  possess  a classical  analog,  the 
usual  semlclassical  statistical  models  for  photodetection  are  not 
applicable  to  TCS  reception.  In  this  paper,  we  present  perform- 
ance results  for  direct,  heterodyne,  and  homodyne  detection  of  TCS 
radiation.  We  shall  show  that  TCS  radiation  offers  a modest  energy 
advantage  (at  most  3 dB)  over  coherent-state  light  In  direct  detec- 
tion of  binary  pulse-position  modulated  digital  signals,  and  in 
heterodyne  detection  of  amplitude  modulated  analog  signals.  A much 
more  significant  improvement  will  be  exhibited  for  the  case  of 
liomodyne  detected  amplitude  modulated  signals.  In  particular,  under 
near-field  propagation  conditions  TCS  homodyne  signal- to-noise  ratio 
will  exceed  that  for  coherent-state  homodyning  by  a factor  of  Ng  + 1, 
where  Ng  is  the  average  number  of  signal  photons.  Under  far-field 
propagation  conditions,  assuming  proper  mode-mixing  with  a TCS  local 
oscillator  in  the  vicinity  of  the  receiver,  homodyne  detection  can 
attenuate  post-measurement  quantum  noise  by  a factor  equal  to  the 
radiative  loss  encountered  in  the  channel.  The  effect  of  finite 
quantum  efficiency  on  these  results  will  be  discussed. 


MAXIMUM  LIKELIHOOD  SEQUENCE  ESTIMATION  FOR  RANDOMLY  DISPERSIVE  OPTI- 
OPTICAL-COMMUNICATION  CHANNELS,  Robert  E.  Morley,  Jr.  (Micro-Term, 
Inc.,  St.  Louis,  Missouri  63117)  and  Donald  L.  Synder  (Department 
of  Electrical  Engineering  and  Biomedical  Computer  Laboratory, 
Washington  University,  St.  Louis,  Missouri  63130).  Receiver  designs 
for  maximum  llklhood  sequence  estimation  of  digital  data  transmission 
through  randomly  dispersive  optical-communication  channels  are 
developed.  The  random  channels  are  constrained  to  have  finite  mem- 
ory in  the  sense  that  the  casual  minimum  mean  square  error  estimate 
of  the  channel  output  at  any  given  time  is  a function  of  only  a 
finite  amount  of  past  data  and  a finite  number  of  information  sym- 
bols. Examples  of  the  channels  included  are  the  time  varying 
Rayleigh,  Ricean  and  lognormal  fading  channels  whose  covariance 
functions  have  finite  support  and  phase  unsynchronized  channels 
which  satisfy  the  finite-memory  constraint.  The  maximum  likelihood 
sequence  estimation  receivers  developed  are  optimal  in  the  sense 
that  they  minimize  the  probability  of  selecting  the  incorrect  mes- 
sage. Optimal  receivers  are  presented  not  only  for  observations 
modeled  by  continuous  processes,  such  as  encountered  with  hetrodyne 
receivers,  but  also  for  observations  modeled  by  point  processes,  such 
as  those  encountered  with  direct-detection  receivers.  The  informa- 
tion sequence  may  be  either  a stream  of  independent,  equally-llkely 
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symbols  of  binary  or  M-ary  alphabets  or  the  output  sequence  of  a 
trellis  or  convolutional  encoder.  An  upper  bound  on  the  average 
message  error-probability  is  derived. 


QUANTUM  OPTIMIZED  RANDOM  CODING  EXPONENT  FOR  (N,R)  BLOCK-CODED 
BINARY  PURE  STATES,  Nam-Soo  Myung  (Department  of  Electrical  Engineer- 
ing and  Computer  Science,  Massachusetts  Institute  of  Technology, 
Cambridge,  Massachusetts  02139).  The  random  coding  bound  for  binary 
(N,R)  block-coded  sequence  of  pure  quantum  states  is  considered. 
Lagrange  multiplier  method  is  employed  to  determine  the  optimum 
detection  operator  at  each  band.  It  is  found  that  the  measurement 
which  minimizes  the  random  coding  baud  is  the  same  as  the  optimum 
measurement  for  detecting  equally  probable  pure  state  binary  signals. 
Comparison  of  random  coding  exponent  among  different  receivers  is 
made  under  signal  energy  and  band  width  constraints. 


MAXIMIZATION  OF  MUTUAL  INFORMATION  AND  MINIMIZATION  OF  DETECTION 
ERROR  IN  QUANTUM  COMMUNICATIONS,  Vincent  W.S.  Chan  (Department  of 
Electrical  Engineering,  Cornell  University,  Ithaca,  New  York  14853). 
In  this  paper,  we  will  consider  the  maximization  of  the  mutual  Infor- 
mation (I(g;J))  between  the  set  of  possible  quantum  system  states 
(g)  and  the  set  of  possible  outcomes  (J)  of  a quantum  measurement. 

The  maximization  is  over  the  choice  of  all  feasible  quantum  measure- 
ments (i.e.,  measurements  characterized  by  operator-valued  measures). 
Explicit  solutions  have  been  found  for  the  binary  pure  state  problem 
and  the  M-ary  problem  with  pairwise  conanutlng  density  operators.  In 
both  cases  the  maximizing  measurement  Is  a 'complete*  measurement 
(i.e.,  the  system  is  left  in  a known  eigenstate  after  measurement). 
Hence  the  number  of  possible  outcomes  N can  be  larger  than  M,  the 
number  of  possible  states.  In  fact  N can  be  as  large  as  M*dlm{H}, 
where  H is  the  Hilbert  Space  spanned  by  the  states  of  the  system. 

We  found,  in  both  cases,  the  measurement  that  maximizes  mutual  in- 
formation also  minimizes  detection  error  probabilities,  although 
the  converse  is  only  true  for  the  binary  pure  state  problem.  For 
the  M-ary  problem  with  pairwise  connutlng  density  operators  the 
measurement  that  maximizes  I(g;J)  has  N - dlm{H)  possible  outcomes, 
with  N possibly  larger  or  less  than  M.  It  can  be  shown  that  the 
measurement  that  minimizes  Pr[c]  may  not  maximize  I(g;J). 


A FAST  EVALUATION  OF  ERROR  RATE  IN  FIBER  OPTIC  DIGITAL  SYSTEMS  WITH 
CORRELATED  SYMBOLS,  Nevio  Benvenuto  and  Sllvano  G.  Pupolln  (Univers- 
ity of  Padova,  Department  of  Electrical  Engineering,  Via  Gradenlgo 
6A,  35100  - PADOVA,  Italy).  A fast  evaluation  of  the  error  rate  in 
fiber  optic  digital  transmission  systems  with  mutually  Independent 
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symbols  has  been  proposed.  The  method  Is  eased  on  a Gram-Char Her 
series  expansion  and  requires  the  joint  moments  of  the  digital  sig- 
nal and  shot  noise  variance  (this  variance  Is  linearly  related  to 
the  data  message). 

The  present  paper  extends  the  above  method  to  systems  with  correlated 
symbols,  where  correlation  Is  produced  by  line  encoding.  Using  a 
sequential  machine  representation  for  line  encoder  a fast  technique 
of  moment  evaluation  Is  set  up,  where  the  computational  work 
Increases  linearly  with  the  number  of  Interferers,  thus  avoiding  the 
exponential  growth  of  exhaustive  method. 
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TRELLIS  CODING  FOR  DEGRADED  BROADCAST  CHANNELS,  W.J.  Leighton,  III 
and  Harry  H.  Tan  (Department  of  Electrical  Engineering  and  Computer 
Science,  Princeton  University),  Coding  theorems  are  proved  for  two- 
receiver  degraded  discrete  memoryless  broadcast  channels  that  use 
superposition  trellis  encoders  and  (1)  Viterbi-algorlthm  type  de- 
coders at  both  receivers  or  (2)  Stack  sequential  type  decoders  at 
both  receivers.  In  the  latter  case,  it  is  also  shown  that  the  ex- 
pected number  of  computations  per  decoded  sublock  is  finite  at  all 
rates  inside  a rate  region  RcoMP  contained  in  the  capacity  region. 
Finally,  RcoMP  is  computed  for  a degraded  binary  symmetric  broadcast 
channel  and  is  shown  to  be  properly  contained  in  the  capacity  region. 


AN  ACHIEVABLE  RATE  REGION  FOR  THE  MULTIPLE-ACCESS  CHANNEL  WITH  FEEDBACK, 
Thomas  M.  Cover  (Department  of  Electrical  Engineering,  Stanford  Uni- 
versity, Stanford,  CA  94305)  and  S.K.  Leung-Yan-Cheong  (Department 
of  Electrical  Engineering,  M.I.T.,  Cambridge,  MA  02139).  An  achiev- 
able rate  region  Rj  S I (Xj ;Y | X2 ,U) , R2  ^ I (X2 ; Y | X^ ,U) , Rj  + R2  ^ 
I(Xi,X2;Y),  is  exhibited  for  the  multiple-access  channel  with  feed- 
back. This  region  exceeds  the  achievable  rate  region  without  feed- 
back and  exceeds  the  rate  point  found  by  Gaarder  and  Wolf  for  the 
binary  erasure  multiple-access  channel  with  feedback. 

The  presence  of  feedback  allows  the  Independent  transmitters  to  under- 
stand each  other's  intended  transmissions  before  the  receiver  has  suf- 
ficient information  to  achieve  the  desired  decoding.  This  allows  the 
transmitters  to  cooperate  in  the  transmission  of  Information  resolv- 
ing the  residual  uncertainty  of  the  receiver.  At  the  same  time,  fresh 
Independent  information  from  the  transmitters  is  superimposed  on  the 
cooperative  correction  information. 


FEEDBACK  CAPACITY  OF  DEGRADED  BROADCAST  CHANNELS,  Abbas  El  Gamal 
(Information  Systems  Laboratory,  Stanford  University,  Stanford,  CA 
94305) . We  consider  the  model  of  one  sender  and  two  receivers  con- 
nected through  two  cascaded  discrete  memoryless  channels  {Pi(Y|X)} 
and  {P2(Z|Y)}.  We  prove  that  the  addition  of  casual  noiseless  feed- 
back from  both  receivers  to  the  sender  does  not  enlarge  the  capacity 
region  of  the  channel.  The  result  is  intuitively  clear  since  feed- 
back merely  adds  degraded  forms  of  the  past  sender  sequence.  It  is 
also  consistent  with  Shannon's  result  on  the  discrete  memoryless 
channel  with  feedback. 

We  next  generalize  Shannon' s entropy  power  inequality  for  the  additive 
white  Gaussian  noise  channel  to  the  case  when  casual  noiseless  feed- 
back is  added.  Then,  using  Hadamard's  Inequality  we  derive  an  upper 
bound  on  the  entropy  of  the  output  sequence  of  the  AWGN  channel  with 
feedback.  The  derived  bounds  are  used  to  prove  that  the  capacity  of 
the  degraded  AWGN  broadcast  channel  is  not  Increased  by  feedback. 
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ON  THE  COMPUTABILITY  OF  AN  ACHIEVABLE  RATE  REGION  FOR  THE  BINARY- 
INPUT  BROADCAST  CHANNEL,  B.E,  Hajek  and  M.B.  Pursley  (Coordinated 
Science  Laboratory,  University  of  Illinois,  Urbana,  Illinois  61801). 
Cover  and  van  der  Meulen  Independently  established  an  achievable 
rate  region,  R , for  a general  discrete  memoryless  broadcast  channel 
that  transmits  separate  messages  at  rates  Rj^  and  R2  to  each  of  the 
two  receivers  and  a common  message  at  rate  Rg  to  both  receivers. 

The  capacity  region  R * satisfies  the  following  Internal  consistency 
condition:  If  (Rj,R2,Rq)  « R*,  Xj  2 0,  ^ 0*  ®nd  Xj^  + X2  s 1 then 

(Rj  + XjRq,  R2  + ^2^0’  “ X2)Rf^)6  R*. 

Our  results  Imply  that  R does  not  satisfy  this  condition  and  can 
therefore  be  enlarged.  Let  R denote  the  smallest  region  which  con- 
tains R and  Is  Internally  consistent  In  the  above  sense. 

The  Inconsistency  of  R Is  deduced  from  Investigation  of  R q and  ^q, 
the  projections  onto  the  Rg  « 0 plane  of  R and , respectively. 

These  correspond  to  the  situation  In  which  there  Is  no  common  mes- 
sage. 

We  show  that’^Q  Is  the  closed  convex  hull  of  the  set  of  all  (R  ,R_) 
such  that 

S I(U^,Uq;Y^)  for  1 - 1,2 

and 


2 


for  some  Markov  chain  (U,X,Y)  such  that  U - (UpU„,Ug)  are  mutually 
independent  and  Y - (Yl,Y2)  is  a broadcast  channel  output  correspond- 
ing to  Input  X.  For  binary-input  channels,  we  show  that  Rg  can  be 
calculated  by  considering  only  those  (U,X,Y)  for  which  Uq  Is  binary, 
Ui  and  U2  are  ternary,  and  X Is  a (deterministic)  function  of  and 
that  Rg  can  be  calculated  by  considering  only  binary  auxiliary  random 
variables.  We  present  examples  for  which  R g is  strictly  larger  than 
Rg  and  hence  ^ is  strictly  larger  than  R . 
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THE  GAUSSIAN  WIRETAP  CHANNEL,  S.K.  Leung-Yan-Cheong  (Electronic  Sys- 
tems Laboratory,  Massachusetts  Institute  of  Technology,  Cambridge, 

MA  02139)  and  Martin  E.  Heilman  (Department  of  Electrical  Engineer- 
ing, Stanford  University,  Stanford,  CA  9A305) . In  a recent  paper, 
Wyner  introduced  the  concept  of  the  wiretap  channel.  He  considered 
the  case  in  which  data  is  to  be  transmitted  reliably  over  a discrete 
memoryless  channel  (referred  to  as  the  main  channel)  to  a legitimate 
receiver.  The  wiretapper  views  the  output  of  the  main  channel  through 
another  discrete  memoryless  channel.  It  is  assumed  that  the  wire- 
tapper  knows  the  encoding  scheme  used  at  the  transmitter  and  the 
decoding  scheme  used  by  the  legitimate  receiver.  The  object  is  to 
design  the  encoder-decoder  so  as  to  maximize  the  transmission  rate 
R to  the  legitimate  receiver  and  the  equivocation  d of  the  data  at 
the  wiretapper. 

In  this  note,  Wyner's  results  are  extended  to  the  Gaussian  wiretap 
channel.  The  R vs.  d curve  is  determined  explicitly  through  the 
use  of  some  special  properties  of  the  Gaussian  channel.  It  is  shown 
that  the  complete  set  of  achievable  (R,d)  pairs  is  given  by 
{(R,d)|R  S Cff,  d S 1,  Rd  i Cg)  where  is  the  capacity  of  the  main 
channel  and  Cg  is  the  difference  between  the  capacities  of  the  main 
and  wiretap  channels. 
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THE  SOURCE  CODING  THEOREM  REVISITED:  A COMBINATORIAL  APPROACH, 
Giuseppe  Longo  (Institute  di  Elettrotecnica  ed  Elettronica,  Universita 
di  Trieste,  Trieste,  and  Centre  International  des  Sciences 
Mecaniques,  Udine,  Italy)  and  Andrea  Sgarro  (Institute  de 
Elettrotecnica  ed  Elettronica,  Universita  di  Trieste,  Trieste,  Italy). 
In  this  paper  a combinatorial  approach  is  proposed  to  the  classical 
source  coding  problems  for  a finite  memoryless  stationary  source 
(achievable  rates  and  error  probability  exponent).  This  approach 
provides  a sound  heuristlcal  justification  for  the  widespread  ap- 
pearance of  entropy  and  divergence  (Kullback's  discrimination)  in 
source  coding.  Use  is  made  of  the  fact  that  asymptotically  the  rate 
of  any  composition  class  is  precisely  an  entropy,  while  its  normal- 
ized probability  exponent  is  precisely  a divergence;  similar  results 
hold  also  for  any  union  of  composition  classes.  (A  composition  class 
is  a set  made  up  by  all  the  source  sequences  which  are  permutations 
of  one  another.)  These  remarks  allow  us  to  derive  the  classical 
results  by  "approximating"  the  relevant  sets  by  means  of  unions  of 
composition  classes.  No  use  is  made  either  of  the  law  of  large  num- 
bers or  of  Chebyshev  inequality. 


SOURCE  CODING  TO  MINIMIZE  THE  PROBABILITY  OF  BUFFER  OVERFLOW,  Pierre 
A.  Humblet  (Electronic  Systems  Laboratory,  Department  of  Electrical 
Engineering  and  Computer  Science,  Massachusetts  Institute  of  Technol- 
ogy, Cambridge,  MA  02139).  A discrete  memoryless  asychronous  source 
emits  symbols  that  are  encoded  one  by  one  and  stored  in  a buffer 
before  being  transmitted  over  a synchronous  output  line.  For  each 
source  code,  the  probability  that  the  arrival  of  a symbol  causes  a 
buffer  overflow  decreases  roughly  exponentially  with  the  size  of 
the  buffer.  If  symbol  i has  probability  pi  and  is  encoded  into  a 
codeword  of  length  mj,  and  if  the  Laplace-Stielt jes  transform  of 
intermission  time  distribution  is  A(s) , the  rate  of  exponential 

sm^ 

decrease  is  related  to  the  supremum  of  the  s such  that  A(s)  Ip . e 

il.  i 


We  develop  an  iterative  algorithm  for  constructing  a prefix  condition 
code  that  maximizes  this  supremum.  The  algorithm  has  two  main  parts. 
In  each  iteration  the  first  part  finds  a prefix  condition  code 
sm. 


minimizing  Ip.e 
1 ^ 


for  a s s 0 given  by  the  second  part  during  the 


previous  iteration;  it  is  a generalization  of  Huffman' s encoding 
procedure.  The  second  part  computes  the  supremum  defined  above  for 
the  code  generated  by  the  first  part.  We  show  that  this  algorithm 
yields  an  optimal  code  in  a finite  time.  We  also  give  an  upper 
bound  on  the  maximum  of  the  supremum. 
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ON  VARIABLE-TO-VARIABLE  CODING  FOR  DISCRETE  MEMORYLESS  SOURCES, 
Leonardus  Th.  N.M.  Mulder  and  David  L.  Cohn  (Electrical  Engineering 
Department,  University  of  Notre  Dame,  Notre  Dame  , Indiana  46556). 

The  objective  of  any  noiseless  source-coding  algorithm  is  to  mini- 
mize the  rate  of  the  coding-scheme  at  some  fixed  level  of  complexity. 
We  consider  a fixed  number  of  code  words  n.  Optimal  variable-to- 
block  and  block-to-varlable  schemes  are  known.  As  can  be  shown,  a 
combination  of  both  schemes  does  not  lead  to  an  optimal  varlable- 
to-variable  scheme. 

For  n S 10  optimal  codes  for  binary  sources  were  found  by  enumeration 
of  different-weighted  binary  trees.  For  n > 10  and  for  non-binary 
sources,  enumeration  becomes  prohibitive.  As  an  alternative,  sub- 
optimal  algorithms  for  finding  good  codes  have  been  developed.  In 
these  algorithms,  'worst ' -nodes  in  the  encoding  tree  are  extended. 

A criterion  for  'worst'-node  is  derived.  The  algorithms  are  com- 
putationally efficient  and  the  resulting  codes  out-perform  the 
combination  of  optimal  varlable-to-block  and  block-to-variable 
schemes  for  most  sources.  For  low  entropy  sources,  the  combination 
scheme  becomes  run-length  encoding  which  turns  out  to  be  optimal. 


BOUNDS  ON  THE  COST  OF  OPTIMAL  UNIQUELY  DECIPHERABLE  CODES,  Norbert 
Cot  and  John  Gill  (Electrical  Engineering  Department,  Stanford 
University,  Stanford,  CA  94305).  Let  {bj,...,bt}  be  the  positive 
costs  of  the  symbols  of  a t-ary  alphabet.  By  the  converse  to 
Shannon's  noiseless  coding  theorem,  a lower  bound  on  the  average 
codeword  cost  for  any  uniquely  decipherable  code  with  codeword 
probabilities  is  H(pi , . . . ,Pn) /log2X , where  H(p^ , . . . ,Pj^)  is  the 
entropy  in  bits  of  the  codeword  ensemble  and  \ is  the  unique  positive 

-b^  -b 

root  of  the  equation  z + ...  + z » 1.  We  generalize  Shannon's 

well  known  technique  for  constructing  nearly  optimal  prefix  codes 
and  obtain  as  an  upper  bound  for  the  cost  of  optimal  uniquely 
decipherable  codes  H(pj , . . . ,p^) /log2X  + b^  + b^  log^(X2)  + b^ 

log^(Aj/X2)  + ...  unique  positive 

-bf  -b 

root  of  the  equation  z + . . . + z • 1 . 


ADAPTIVE  HUFFMAN  CODES,  Robert  G.  Gallager  (Massachusetts  Institute 
of  Technology,  Department  of  Electrical  Engineering  and  Computer 
Science  and  Electronic  Systems  Laboratory).  In  honor  of  the  twenty- 
fifth  anniversary  of  Huffman  Coding,  three  new  results  about  Huffman 
codes  are  presented.  The  first  result  shows  that  a binary  prefix 
condition  code  is  a Huffman  code  (with  a convention  of  assigning  0 
to  the  more  probable  branch  in  the  construction  procedure)  if  the  proba- 
bilities of  the  intermediate  and  terminal  nodes  in  the  code  tree  are 
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lexicographically  ordered.  The  second  result  upper  bounds  the 
redundancy  (expected  length  minus  entropy)  of  a binary  Huffman  code 
by  PjL  + log2(2(log2e)/e]  " + .086  where  is  the  probability  of 

the  most  likely  source  letter.  The  third  result  is  a simple  algorithm 
for  adapting  a Huffman  code  to  slowly  varying  estimates  of  the  source 
probabilities.  In  essence,  one  maintains  a running  count  of  uses  of 
each  node  in  the  code  tree.  Whenever  the  occurrence  of  a message 
increases  a node  count  above  the  count  of  the  next  lower  node  in 
the  lexicographic  ordering,  the  nodes,  with  their  attached  subtrees, 
are  interchanged. 


AN  EQUAL-LENGTH-AT-OUTPUT  UNIVERSAL  METHOD  OF  CODING,  B.  Fitingof 
(Technion-Israel  Institute  of  Technology,  Haifa,  Israel).  Three 
different  concepts  of  optimal  coding  method  are  defined  and  com- 
pared: The  asymptotically  optimal  method  of  coding  (AOMC) , the 
asymptotically  optimal  on  the  average  method  of  coding  (AOAMC)  and 
the  universal  method  of  coding  for  a class  of  sources  (UMC) . A 
sufficient  condition  of  universality  for  the  class  of  Bernoullian 
sources  with  a given  alphabet  is  found  in  terms  of  quasi-entropy  of 
input  words.  The  notion  of  a monotone  source  is  Introduced  and 
used  in  the  construction  of  a called  K-method,  where  input  words 
of  various  lengths  are  encoded  in  output  words  of  equal  length.  It 
is  proved  that  the  K-method  is  an  AOMC  for  any  monotone  sources,  in 
particular,  for  a Bernoullian  source.  A special  case  of  the  K- 
method  is  considered  and  proved  to  be  a UMC  for  the  class  of  all 
Bernoullian  sources  with  a given  size  of  alphabet.  An  algorithm 
implementing  the  K-method  of  coding  is  suggested,  which  requires 
very  low  storage. 
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ALGEBRAIC  COMPLEXITY  OF  COMPUTATION,  S.  Winograd  (IBM  Research, 
Yorktown  Heights,  NY).  This  talk  will  survey  some  recent  results  in 
the  theory  of  algebraic  complexity  of  computation  with  particular 
emphasis  on  problems  connected  with  signal  processing  and  coding. 
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COMPUTATIONAL  COMPLEXITY  OF  CONVOLUTIONS  EVALUATED  BY  NUMBER 
THEORETIC  TRANSFORMS,  H.  Nussbaumer  (IBM  CER,  06610  La  Gaude,  France). 
Various  Number  Theoretic  Transforms  (NTT)  have  been  introduced  re- 
cently as  a means  of  reducing  the  computational  complexity  of  con- 
volutions and  correlations.  In  this  paper,  we  make  precise  the 
processing  requirements  for  pseudo  Mersenne  and  pseudo  Fermat  num- 
ber transforms.  These  results  are  compared  to  those  corresponding 
to  Mersenne  and  Fermat  number  transforms  and  extended  to  cover  the 
case  of  complex  transforms. 


The  computational  complexity  of  non  recursive  digital  filters  imple- 
mented with  taese  various  transforms  is  evaluated  and  compared, 
under  certain  simplifying  assumptions,  to  that  corresponding  to 
direct  computation.  We  show  that  the  processing  efficiency  of  the 
NTT  approach  increases  with  the  required  precision  and  that  optimum 
results  are  obtained  with  Fermat  number  transforms  and  the  complex 
pseudo  Mersenne  and  pseudo  Fermat  number  transforms  defined  modulo 


2 


(2^^  -1) 
2'’-! 


and  modulo 


2 

(2^^  +1) 
2'’-! 


with  q prime. 


The  computational  complexity 


as  a function  of  filter  length  is  investigated  and  it  is  shown  that 
one-dimensional  filters  having  a length  of  up  to  about  300  taps  can 
be  Implemented  efficiently  with  NTT's. 


FAST  ALGORITHMS  FOR  MULTI-VARIABLE  AND  MULTI-DIMENSIONAL  SYSTEMS, 

M.  Morf  (Information  Systems  Laboratory,  Stanford  University,  Stan- 
ford, CA  9A305) . The  computational  complexity  and  efficiency  of 
algorithms  for  the  analysis,  design  and  signal  processing  involving 
single  and  multi-variable  and  multl-dlmenslonal  systems  depends 
strongly  on  the  structure  of  such  systems.  Most  algorithms  are 
based  on  matrix  multiplications,  factorizations  or  inversions.  The 
structure  of  the  underlying  systems  is  reflected  in  the  matrix 
representations  of  these  operations.  Often  the  structure  of  matrices 
is  visible  by  inspection.  For  Instance  the  Toeplitz  or  Hankel 
matrices  reflect  statlonarity  of  signals  or  tlme-invarlance  of  sys- 
tems; the  matrix  entries  are  functions  of  the  difference  or  the  sum 
of  the  indices.  However,  in  general,  operations  such  as  sums  of 
products  or  Inverses  of  matrices  with  an  obvious  structure  often  do 
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not  show  a particular  pattern.  If  the  structure  can  be  exposed, 
very  likely  it  can  be  used  to  reduce  the  computational  complexity 
and  number  of  operations,  l.e.,  increase  the  efficiency  of  algorithms 
for  matrix  calculations. 

A particular  class  of  considerable  interest  to  the  signal  processing 
and  systems  area  are  the  matrices  that  can  be  Interpreted  as  sub- 
matrices of  the  coefficient  matrix  of  rational  polynomials  in  two 
or  more  variables  ("2  or  M-D  transforms") . Sums  of  products  of 
Toeplltz  or  Hankel  matrices  represent  one  particular  case,  where 
the  denominator  of  the  associated  "2-D  transform"  is  just  the  dif- 
ference of  the  two  polynomial  variables.  A special  case  of  this 
are  the  Christophel-Darboux  formulas  for  the  resolvent  associated 
with  a Toeplitz  kernel.  Another  particular  case  of  recent  interest 
are  the  variable  Fast  Fourier  and  related  transforms , e.g . , the 
Chirp  Z-Transform  corresponds  to  a factorization  of  the  FFT  matrix 
into  a product  of  Diagonal  and  Toeplitz  matrices.  A crucial  property 
of  these  matrices  is,  however,  that  they  can  be  factored  into 
Kronecker  products  of  simpler  matrices;  these  products  in  turn 
correspond  to  (primitive)  factors  of  the  associated  "2-D  transform". 

We  have  now  algorithms  that  can  find  rational  approximations  of 
functions  in  two  variables  efficiently,  thus  a method  to  expose 
the  structure  of  a given  matrix.  The  area  of  2-D  (matrix)  polynomials 
has  recently  grown  very  rapidly  and  the  newly  available  results  can 
shed  light  on  the  connections  between  the  various  matrix  results, 
the  associated  fast  algorithms  and  their  complexity.  A survey  of 
these  results  and,  in  particular,  relations  between  displacement 
matrices  (Toeplitz  and  Hankel)  and  FFT  algorithms  will  be  presented. 


THE  USE  OF  DECISION  TREES  IN  COMPUTATIONAL  COMPLEXITY,  Andrew  C.  Yao 
(Department  of  Computer  Science,  Stanford  University,  Stanford,  CA 
94305).  For  a given  computational  problem,  its  complexity  is  under- 
stood through  devising  efficient  algorithms  to  find  solutions,  on  the 
one  hand,  and  proving  lower  bounds  to  the  number  of  steps  required  by 
all  algorithms,  on  the  other.  In  discussing  lower  bounds,  one. must 
first  adopt  a model  suitable  for  representing  all  possible  algorithms 
under  consideration.  The  decision  tree  model  has  proven  to  be 
appropriate  for  a great  varlety'of  problems,  such  as  information 
retrieval,  string  matching,  finding  shortest  paths,  etc.,  and  many 
fruitful  results  obtained.  In  this  talk,  we  present  an  overview  of 
the  decision  tree  model.  Examples  are  discussed  to  illustrate  the 
interesting  techniques. 
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FAST  STATISTICAL  ALGORITHMS,  M.  Shamos  (Department  of  Computer 
Science,  Carnegie-Mellon  University,  Pittsburgh,  PA  15213).  We  will 
discuss  techniques  for  designing  efficient  algorithms  for  statistical 
computation,  taking  examples  from  nonparametric  estimation,  correla- 
tion, regression  and  time  series  analysis.  We  will  stress  on-line 
methods  and  average-case  analysis  in  developing  a set  of  fast  com- 
putational tools  that  can  be  used  to  build  more  sophisticated 
algorithms.  Attendees  will  be  encouraged  to  pose  problems  that 
they  have  encountered  in  applications. 
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NONPARAMETRIC  ESTItlATION  WITH  LOCAL  RULES,  C.S.  Penrod  and  T.J. 

Wagner  (Department  of  Electrical  Engineering,  University  of  Texas, 
Austin,  Texas  78712).  The  application  of  nearest  neighbor  rules 
and  other  local  rules  to  the  problem  of  estimating  a parameter  6 
is  investigated.  It  is  assumed  that  a loss  function  L,  an  observed 
random  vector  X,  and  data  consisting  of  a sequence  of  independent 
random  vectors  (Xj ,6^) , . . . , (X^,0n)  with  the  same  distribution  as 
(X,6)  are  given.  Conditions  are  shown  for  which,  if  R*  denotes  the 
Bayes  risk  (the  minimum  expected  loss  possible),  then  the  conditional 
expected  loss  of  the  k-nearest  neighbor  rule,  conditioned  on  the  data, 
converges  to  (I  + I/k)R*  for  squared-error  loss  functions.  For  kp- 
nearest  neighbor  rules  where  kp  ® and  kp/p  -♦  0,  conditions  are 
given  under  which  the  rules  are  asymptotically  optimal. 

In  addition,  methods  of  estimating  the  conditional  risk  of  a rule 
with  a particular  data  set  are  Investigated.  For  a class  of  rules 
called  local  rules,  the  performance  of  two  different  estimates  of 
the  risk  is  bounded  independently  of  the  underlying  distribution 
of  (X,6).  This  enables  the  statioticlan  to  construct  confidence 
intervals  for  the  risk  of  the  rule  and  data  he  is  using,  without 
knowledge  of  the  distribution  of  (X,0). 


ROBUST  RANDOM  PARAMETER  ESTIMATION  AND  MINIMUM  FISHER  INFORMATION, 

R.  Doraiswami  (Coppe  - Universldade  Federal  Do  Rio  De  Janeiro, 

Brazil).  An  estimation  of  random  parameters  from  a linear  measure- 
ments model  is  proposed,  when  the  a priori  statistics  are  Incomplete 
and  only  a small  number  of  data  points  are  available.  The  probability 
distributions  of  the  random  variables  are  assumed  unknown.  The  only 
available  statistics  are  the  covariance  of  the  random  parameters 
and  the  'partial  covariance'  of  the  measurement  random  variables. 

Using  min-max  theory,  the  optimal  estimator  is  shown  to  be  a 
sof  t-liraiter . 


ROBUST  WIENER  FILTERS,  Tong  Leong  Lim  and  Saleem  A.  Kassam  (Moore 
School  of  Electrical  Engineering,  University  of  Pennsylvania, 
Philadelphia,  PA  19104).  Linear  filtering  of  signals  in  additive 
noise  is  considered  when  the  power  spectral  densities  (psd's)  of 
the  signal  and  noise  processes  are  not  completely  specified.  Two 
models  for  classes  of  the  psd's  are  defined,  the  e-model  for  con- 
taminated nominal  psd's  and  the  band-model  with  upper  and  lower 
bounds  on  the  psd's.  For  the  minimum  mean-squared-error  criterion, 
filters  which  are  saddlepoint  solutions  for  performance  over  the 
two  classes  of  psd's  are  obtained;  the  saddlepoint  or  most  robust 
filters  are  optimum  for  least  favorable  pairs  of  signal  and  noise 
psd's  which  have  shapes  tending  to  make  them  look  alike.  An 
example  is  presented  illustrating  the  nature  of  the  robust  solution 
and  the  least  favorable  psd's,  and  showing  the  usefulness  of  the 
most  robust  filter  in  maintaining  its  performance  over  classes  of 
psd's. 
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ON  COUNTABLY  INFINITE  HYPOTHESIS  TESTING,  Jack  Koplowltz  (Department 
rf  Electrical  and  Computer  Engineering,  Clarkson  College  of  Technol- 
ogy, Potsdam,  NY  13676).  Let  X],X2,...  be  a sequence  of  independent 
Identically  distributed  Bernouilli  random  variables  with  unknown 
parameter  p = Pr{xj^=l}.  Let  S = be  a countable  subset 

e [0,l].  Consider  the  countable  set  or  hypotheses  Hj^  : p“Pj^,  i*l,2, 

. . . together  with  the  null  hypothesis  Hg  : p i S.  Cover  has  demon- 
strated a decision  procedure  which  makes  only  a finite  number  of 
mistakes  with  probability  one  in  determining  the  true  hypothesis  for 
any  p e [0,l]  - Ng,  where  Ng  is  a set  in  [0,l]  - S of  Lebesgue 
measure  zero.  The  procedure  generalizes  to  determining  the  mean 
with  unknown  distribution  but  finite  variance. 

Consider  the  hypothesis  test  Hg  : p is  irrational  vs.  Hj  : p is 
rational.  It  has  previously  been  shown  that  for  any  procedure 
which  converges  in  probability  under  Hp  there  exists  an  uncount- 
able set  of  irrationals  Ng,  such  that  if  p e Ng  a zero  limiting 
probability  of  error  cannot  be  achieved. 

On  the  other  hand  consider  S = {pj,p2,...},  Pj^  = 1/2^.  It  can  be 
shown  that  the  hypothesis  test  Hj  : i*l,2,...,  vs.  Hg  : p ^ S can 
be  resolved  for  all  p e [0,l].  In  this  paper  we  obtain  necessary 
and  sufficient  conditions  for  which  the  countable  infinite  hypothesis 
test  can  be  resolved  for  all  p e [0,l].  It  is  shown  that  the  true 
hypothesis  can  be  resolved  for  all  p e [0,l]  if  and  only  if  the 
closure  of  the  set  S is  countable. 


A TEST  FOR  STATIONARITY , David  J.  Thomson  (Bell  Laboratories, 
Whippany,  NJ  07981).  A test  for  stationarity  of  a time  series  is 
given  which  has  the  properties  of  excellent  performance  in  practice, 
simplicity,  and  modest  computational  requirements.  The  test  pro- 
cedure consists  of  three  steps:  Direct  estimation  of  the  spectrum 
on  subsets  of  the  data,  comparison  of  the  subset  spectrum  estimates 
at  each  frequency  using  the  Bartlett  M statistic  and  finally,  test- 
ing the  values  of  M for  conformance  to  their  null  distribution 
using  the  one-sided  Kolmogorov  stastic  D“.  This  procedure  is  sensi- 
tive to  changes  localized  either  in  time  or  frequency,  and  also 
indicates  v/hen  excessively  short  subsets  are  chosen.  It  is  shown 
that  modest  gains  in  sensitivity  may  be  achieved  by  prefiltering 
the  data  but  that  smoothing  the  spectral  estimates  Increases  the 
probability  of  type  I errors.  A useful  approximation  for  the  power 
of  the  test  is  described  and  examples  are  given  for  both  stationary 
and  nonstationary  series. 


64 


SESSION  C2 


FISHER  INFORM.VTION  OF  ORDER  S,  D,E,  Boekee  and  Y.  Boxma  (Laboratory 
for  Information  Theory,  Department  of  Electrical  Engineering,  Delft 
University  of  Technology,  4 Mekelweg,  Delft,  The  Netherlands).  The 
Fisher  information  is  a well-known  measure  for  the  Information  about 
a parameter  which  is  contained  in  observations. 

It  satisfies  some  basic  properties  which  any  information  measure 
should  satisfy.  Its  main  application  lies  in  estimation  theory  since 
the  Fisher  information  can  be  used  to  obtain  a lower  bound  on  the 
accuracy  of  parameter  estimators  in  terms  of  their  variance.  Further- 
more, it  is  known  that  minimum  variance  estimator  for  a location 
parameter  must  have  a gaussian  distribution,  in  which  case  the 
Fisher  information  is  minimized.  However,  it  is  well-known  that 
the  variance  is  not  always  a suitable  criterion  for  the  performance 
of  estimators. 

In  this  paper  we  consider  a generalization  of  the  Fisher  information, 
called  the  Fisher  information  of  order  s.  It  is  based  on  arbitrary 
powers  of  the  absolute  value  of  the  derivative  of  the  log  likelihood 
function.  This  includes  the  Fisher  information,  if  we  consider 
second  powers. 

We  shall  consider  some  basic  properties  of  this  measure  from  an  in- 
formation theoretic  point  of  view.  It  will  be  related  to  some  other 
information  measures. 

It  will  be  shown  that  this  Fisher  information  can  be  used  to  obtain 
bounds  on  the  s-th  absolute  central  moment  of  parameter  estimators, 
with  s i 1.  We  Introduce  the  notion  of  MSB  estimators  and  obtain  a 
class  of  density  functions,  called  the  exponential  power  distribution, 
for  which  MSB  estimators  contain  minimal  Information.  The  bound  ob- 
tained will  be  compared  with  other  bounds  on  the  s-th  absolute  cen- 
tral moment  which  are  known  in  the  literature.  We  shall  also  give 
results  for  the  case  that  the  parameter  to  be  estimated  is  a random 
parameter . 

We  shall  consider  location  and  scale  parameters  as  special  cases  and 
obtain  some  numerical  results  for  the  Fisher  information  of  order  s. 

It  is  possible  to  extend  the  notion  of  entropy  power  to  the  entopy 
moment  of  order  s.  We  shall  show  that  there  exists  an  interesting 
analogy  between  the  Fisher  information  of  order  s and  the  entropy 
power  of  order  s. 

Finally  we  shall  discuss  some  applications  of  the  Fisher  information 
or  order  s to  parameter  estimation  problems  in  additive  noise. 
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DATA  SMOOTHING  VIA  ORDER  STATISTICS,  Shu-gwel  Tyan  (Polytechnic 
Institute  of  New  York,  Brooklyn,  NY  11201).  Through  local  monotonic 
regression  an  example  is  used  to  Illustrate  the  optimality  of  some 
simple  median-based  smoothers.  Properties  of  running  medians,  the 
building  blocks  of  median-based  smoothers,  are  explored  by  studying 
their  fixed  points.  They  are  found  to  belong  to  two  classes:  the 
first  class  contains  the  locally  monotonic  sequences  and  the  second 
contains  only  duo-valued  sequences.  Combinations  of  running  medians 
of  various  lengths,  in  parallel  and  in  serial,  are  also  studied. 
Locally  monotonic  sequences  which  come  naturally  with  running 
medians  are  shown  to  be  closely  related  to  other  smoothers  based  on 
order  statistics.  If  the  upper  and  lower  envelopes  of  a data 
sequence  are  defined  as  the  local  minmax  and  the  local  maxmin  respec- 
tively, then  it  can  be  shown  that  the  lower  envelope  of  the  upper 
envelope  is  locally  monotonic.  The  same  is  true  for  the  upper  enve- 
lope of  the  lower  envelope.  Furthermore,  the  running  median  of  the 
proper  length  always  lies  in  between  the  two  locally  monotonic  en- 
velopes, thus  they  can  be  taken  intuitively  as  the  upper  and  the 
lower  estimates. 
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FREE  DISTANCE  PROPERTIES  FOR  A NEW  CLASS  OF  TRELLIS  PHASE  CODES, 

J.B.  Anderson  and  D.P.  Taylor  (Department  of  Electrical  Engineering 
and  Communications  Research  Laboratory,  McMaster  University,  Hamil- 
ton, Ontario,  Canada,  L8S  4L7).  Consider  a constant-amplitude 

phase-varying  signal  of  the  form  ■^2E/T  cos((a)c+fin)t+en)  , (n-1) 

T<tSnT.  is  a frequency  offset  chosen  subject  to  0sJ1j,T<2ti<<Uj,T , 

constant  during  any  T-length  interval.  0j^  is  chosen  to  make  the 
(n+l)-st  frequency  change  continuous  in  phase.  By  varying  the  off- 
set frequency  in  response  to  an  information  sequence,  one  instruments 
a code  in  phase  and  time,  which  can  provide  a much  reduced  error  rate 
over  ordinary  PSK  and  FSK  in  the  same  (or  less)  bandwidth.  We  pre- 
sent a useful  class  of  such  codes,  the  "multi-h"  codes,  and  analyze 
their  error  performance.  In  these  codes,  the  carrier  frequency 
offset  is  either  2iTil/q  or  0,  depending  on  the  information  bit, 
where  i and  q are  integers.  The  phase  transitions  possible  with 
such  a code  form  an  endlessly  repeating  trellis  phase  structure; 
as  such,  only  a limited  number  of  phases  and  phase  transitions  must 
be  recognized  and  demodulated  during  decoding,  and  a simple  Viterbi 
algorithm  scheme  may  be  used.  We  calculate  the  free  distance  of  all 
useful  codes,  that  is,  codes  with  constraint  lengths  1-4,  which 
utilize  20  or  fewer  different  phases.  Two-dimensional  dynamic  pro- 
gramming must  be  used,  because  the  distances  to  phase  trellis 
neighbors  depend  strongly  on  the  Information  sequence.  To  verify 
these  results,  and  to  study  burst  errors,  we  have  tested  a software- 
implemented  encoder/decoder  on  a simulated  Gaussian  channel.  A 3-4 
dB  coding  gain  was  found.  Comments  on  spectral  occupancy  and  a 
comparison  to  soft-decision  decoding  conclude  the  paper. 


AN  ANALYSIS  OF  SEQUENTIAL  DECODING  BASED  ON  CODE  DISTANCE  PROPERTIES, 
P.R.  Chevillat,  (IBM  Research  Laboratory,  CH-8803  RUschlikon, 
Switzerland),  and  D.J.  Costello,  Jr.,  (Department  of  Electrical 
Engineering,  Illinois  Institute  of  Technology,  Chicago,  IL  60616). 

The  computational  effort  and  the  error  probability  of  sequential  de- 
coding with  fixed  (time-invariant)  convolutional  codes  are  analyzed 
without  employing  any  of  the  traditional  random  coding  arguments. 

An  upper  bound  on  the  computational  distribution  P(Cj.  > N^.)  for  a 
specific  fixed  code  is  presented  which  decreases  exponentially  with 
the  code's  column  distance.  It  is  proved  that  rapid  column  distance 
growth  minimizes  the  decoding  effort  and  therefore  also  the  probabil- 
ity of  decoding  failure  (erasure).  In  an  analogous  way  the  undetected 
error  probability  of  sequential  decoding  with  a specific  fixed  code 
is  proved  to  decrease  exponentially  with  the  free  distance  and  to 
increase  linearly  with  the  number  of  minimum  free-welght  codewords. 

The  two  theorems  prove  that  code  construction  for  sequential  decoding 
should  maximize  column  distance  growth  and  free  distance  in  order  to 
guarantee  fast  decoding,  a minimum  erasure  probability,  and  a low 
undetected  error  probability. 
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BRjVNCHING  processes  and  sequential  decoding,  David  Haccoun  (Depart- 
ment of  Electrical  Engineering,  Ecole  Poly technique  de  Montreal, 
Montreal,  Canada).  Modelling  the  incorrect  paths  explored  by  a 
sequential  decoder  as  a special  branching  process  with  two  absorbing 
barriers,  this  paper  presents  a procedure  to  bound  the  average  num- 
ber of  computations  per  decoded  bit.  Using  the  same  model  a bound 
on  the  average  population  of  the  live  incorrect  paths  existing  at 
any  depth  away  from  the  correct  parent  node  will  also  be  presented. 
These  bounds  are  expressed  in  terms  of  the  set  of  incorrect  branch 
metrics  actually  used  by  the  decoder,  and  in  terms  of  the  set  of 
the  stationary  probabilities  of  Massey's  Markov  chain  model  of  the 
metric  differences  on  the  correct  path. 


MODIFIED  VITERBI  DECODER  FOR  BURST  CHANNELS,  Jose  Roberto  B.  de  Marca 
and  R.A.  Scholtz  (Department  of  Electrical  Engineering,  University  of 
Southern  California,  Los  Angeles,  CA  90007).  This  paper  considers 
the  decoding  of  convolutional  codes  transmitted  over  a finite-state 
burst  channel,  by  applying  Vlterbi's  algorithm  to  path  estimation  in 
a combined  encoder-channel  trellis  diagram.  Computer  simulation  is 
used  to  compare  performance  of  this  modified  algorithm  to  a standard 
Vlterbl  algorithm,  for  several  choices  of  two-site  burst  channels 
and  convolutional  codes.  An  upper  bound  to  the  performance  of  the 
modified  decoder  can  be  developed  with  the  aid  of  flow-graph  tech- 
niques, while  simulation  results  for  a "magic  genie"  decoder  supply 
lower  bound  information  for  all  possible  decoders  operating  in  this 
situation. 

Complexity  of  the  modified  decoder  grows  linearly  with  the  number  of 
states  in  the  channel  model.  However,  the  magic  genie  decoder  does 
not  have  this  problem  since  information  concerning  the  true  channel 
state  is  supplied  to  it  by  an  external  source  (the  "genie").  Re- 
placement of  the  genie  by  a channel-state  estimator  is  possible. 

This  provides  an  interesting  way  to  link  Interleaved  decoders,  for 
the  purpose  of  sharing  burst  location  information. 


HADAMARD  TRANSFORM-HAMMING  DISTANCE  DECODING  RULE  FOR  CONVOLUTIONAL 
CODES,  Ledlnh  Chon  Tam  (Centre  de  Recherche  Industrielle  du  Quebec, 
245  Boul.  Hymus,  Pointe-Claire , Quebec,  Canada)  and  Jean-Pierre  Adoul 
and  Roger-Y.  Goulet  (Communication  Research  Center,  University  of 
Sherbrooke,  Sherbrooke,  Quebec,  Canada).  A new  recursion  procedure 
is  derived  for  the  decoding  of  binary  concolut ional  codes  which 
minimizes  the  state  Hamming  distance.  The  procedure  hinges  on  the 
use  of  Hadamard  characteristic  function  in  the  Bayes  estimation  con- 
text. The  structure  of  the  decoder  is  shown  to  be  fully  modular 
and  expandable  and  is  illustrated  for  rate  1/2  and  2/3  "good"  codes. 
Simulation  performed  on  the  binary  symmetric  channel  shows  that  for 
fixed  delay,  the  performances  of  this  decoding  rule  is  slightly 
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better  than  that  of  the  Vlterbl  algorithm.  For  practical  situations 
the  decoder  can  be  implemented  without  "true"  multiplications  and  it 
is  therefore  suited  for  short  codes  with  middle  rates  (e.g.:  1/2, 

2/3,  3/4)  specially  when  hard  decision  is  necessary  as  in  high  speed 
applications. 


PUNCTURED  R = (n-I)/n  CONVOLUTIONAL  CODES  FOR  SIMPLIFICATION  OF 
M/VXlMlfM  LIKELIHOOD  DECODING,  J.  Bibb  Cain,  George  C.  Clark,  Jr., 
and  John  M.  Geist  (Harris  Electronic  Systems  Division,  Melbourne, 
FL) . When  rate  -1/n  codes  are  decoded  using  the  Viterbl  algorithm, 
each  code  state  has  2 paths  entering  it  for  comparison  of  the 
respective  metrics.  However,  with  rate  -(n-l)/n  codes  (n>3)  there 
are  2t>"l  paths  entering  each  state.  Therefore,  the  resulting  com- 
parison and  selection  of  the  path  with  the  best  metric  is  made  much 
more  difficult.  This  will  require  either  much  more  complex  hard- 
ware in  a high  speed,  fully  parallel  machine  or  more  comparisons  in 
a low-speed,  serial  machine.  However,  this  problem  can  be  avoided 
entirely  by  a judicious  choice  of  code  generator  polynomials.  By 
doing  this  one  can  decode  the  code  just  as  one  would  decode  a rate 
1/2  code  with  very  little  additional  complexity.  Moreover,  this 
technique  provides  a straightforward  way  of  building  a selectable 
rate  decoder.  The  codes  are  constructed  by  periodically  puncturing 
suitable  lower  rate  codes,  e.g.,  a R = 2/3  code  can  be  obtained  by 
deleting  every  fourth  bit  at  the  output  of  a R = 1/2  encoder.  The 
resulting  R » 2/3  code  could  be  decoded  a the  original  R = 1/2 
code  with  unreliable  decisions  (or  erasures)  inserted  in  place  of 
the  deleted  bits.  The  Implementation  advantages  of  the  punctured 
code  approach  are  discussed.  In  addition,  we  show  Plotkin-type 
upper  bounds  on  free  distance  for  this  class  of  codes  and  compare 
them  with  similar  bounds  for  general  R = (n-l)/n  codes.  A search 
for  the  best  R = 2/3  (to  k=I0)  and  3/4  (to  k=ll)  codes  of  this  type 
has  been  made,  and  the  best  codes  are  tabulated.  It  is  shown  that 
at  these  rates  and  constraint  lengths  one  can  almost  always  find  a 
punctured  code  with  a minimum  free  distance  as  good  as  the  best 
R » 2/3  and  3/4  codes.  Performance  curves  for  these  codes  are  also 
given. 


SOME  TRANSPARENT  CONVOLUTIONAL  CODES  WITH  A SIMPLE  ENCODER  INVERSE, 
Erik  Paaske  (Institute  of  Circuit  Theory  and  Telecommunication, 
Technical  University  of  Denmark,  DK-2800  Lyngby,  Denmark)  and  Rolf 
Johannesson  (Department  of  Automata  and  General  Systems  Sciences, 
University  of  Lund,  S-220  07  Lund  7,  Sweden).  As  a counterpart  to 
quick-look-in  (QLI)  convolutional  codes,  which  are  not  transparent, 
we  introduce  rate  r » 1/2  easy-look-in-transparent  (ELIT)  codes 
with  a feedforward  Inverse  (1+D,D).  We  present  an  extensive  list 
of  ELIT  codes  with  an  optimum  generalized  distance  profile  d * 
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[dg.dj , . . . ,d^_2] , where  v is  the  constraint  length  and  dj  is  the 
order  j column  distance.  In  general  ELIT  codes  have  doo  superior  to 
that  of  QLI  codes. 
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DIGITAL  CODING  OF  ANALOG  WAVEFORMS,  Herbert  Gish  (Bolt  Beranek  and 
Nevnnan,  Inc.,  50  Moulton  Street,  Cambridge,  MA  02138).  The  digital 
coding  of  analog  waveforms  Is  considered  from  two  aspects.  One 
aspect  Is  the  evaluation  of  theoretical  performance  limits  and  the 
other  is  the  design  of  efficient  coding  systems.  Primary  considera- 
tion is  given  to  waveform  transmission  over  the  bandllmited,  additive 
noise,  Gaussian  channel. 

It  is  shown  how  rate  distortion  theory  can  be  employed  to  obtain 
useful  performance  bounds  on  systems  which  are  required  to  operate 
over  a range  of  channel-to-nolse  ratios.  The  results  obtained  can 
provide  information  concerning  the  variation  of  system  performance 
and,  in  particular,  give  quantitative  information  about  the  threshold 
effect.  In  addition  a method  of  waveform  coding  is  investigated 
which  takes  into  account  the  differences  in  significance  of  the  bits 
used  in  the  binary  description  of  the  quantized  samples  of  the  wave- 
forms. The  performance  of  this  technique  is  compared  to  theoretical 
limits  and  to  PCM  performance.  Consideration  is  given  to  the  trade- 
off between  threshold  extension  and  the  performance  at  high  slgnal- 
to-noise  ratios. 


ASYMPTOTICALLY  OPTIMAL  BLOCK  QUANTIZATION,  Allen  Gersho  (Bell  Labora- 
tories, Murray  Hill,  NJ  0797A) . In  1948  VJ.R.  Bennett  used  a compand- 
ing model  for  nonuniform  quantization  and  proposed  the  formula 

D = --—A-  I p(x)[E'(x)]  ^dx 

12N  J 

for  the  mean-square  quantizing  error  when  N is  the  number  of  levels, 
p(x)  is  the  probability  density  of  the  input,  and  E' (x)  is  the  slope 
of  the  compressor  curve.  The  formula  is  an  approximation  based  on 
the  assumption  that  the  number  of  levels  is  large  and  is  a useful 
tool  for  analytical  studies  of  quantization. 

This  note  gives  a heuristic  argument  generalizing  Bennett's  formula 
to  block  quantization  where  a vector  of  random  variables  is  quantized. 

The  approach  is  again  based  on  the  asymptotic  situation  where  N the  number 
of  quantized  output  vectors  is  very  large.  Using  the  resulting  formula, 
an  optimization  is  performed  leading  to  an  expression  for  the  mini- 
mum quantizing  noise  attainable  for  any  block  quantizer  of  a given 
block  size  k.  The  result  specializes  to  knovm  results  for  the  one- 
dimensional  case  (k-1)  and  for  the  case  of  infinite  block  length 
(k->”) . The  same  heuristic  approach  also  gives  an  alternate  deriva- 
tion of  a bound  of  Ellas  for  multidimensional  quantization. 
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DIFFERENTIAL  ENCODING  FOR  BINARY  MARKOV  SOURCES,  George  Thomas 
(Indian  Institute  of  Science,  Bangalore,  India).  A binary  first 
order  autoregressive  process  and  Its  underlying  memoryless  state- 
transition  process  have  the  same  rate-distortion  function  with 
respect  to  the  probablllty-of-error  criterion,  for  low  values  of 
distortion.  In  this  paper  we  explore  the  possibility  of  encoding 
the  memoryless  process  at  a suitable  rate  R and  distortion  D and 
of  deriving  therefrom  an  (R,D)-approxlmatlon  to  the  output  of  the 
Markov  source.  The  motivation  Is  that  It  Is  then  possible  to  apply 
the  techniques  for  source  encoding  of  memoryless  sources  directly 
to  the  case  of  autoregressive  sources  driven  by  memoryless  processes. 
It  Is  Immediately  seen  that  the  direct  expansion  of  the  encoded 
memoryless  process  using  a feedback  shift  register  circuit  leads  to 
unbounded  error  propagation.  We  propose  a simple  modification  to 
the  source  encoder  which  generates  error  patterns  which  do  not 
propagate.  The  rate-distortion  performance  attainable  with  the 
modified  encoder  Is  evaluated. 


RUN-LENGTU  ENCODING  WITH  FIDELITY  CRITERION  FOR  BINARY  MARKOV 
SOURCES,  P.K.S.  Wah  (Swiss  Federal  Institute  of  Technology,  ETH- 
Zentrum,  8092  Zurich,  Switzerland).  Several  simple  source  encoding 
algorithms  for  binary  stationary  sources,  both  symmetric  and 
asymmetric,  with  memory  are  described.  One  method  Is  to  sample  one 
of  every  M (M>1)  symbols  and  then  to  code  these  selected  symbols 
with  run-length  code.  At  the  receiving  end  the  missing  symbols  are 
reconstructed  according  to  the  dependence  between  neighboring 
symbols.  The  resulting  distortion  for  Markov  sources  of  different 
orders  can  readily  be  calculated.  Other  methods  include  the  ex- 
changing of  I's  to  O's  or  vice  versa  in  the  binary  data  to  form 
long  runs,  so  that  isolated  symbols,  short  runs  or  some  certain  run- 
lengths  are  eliminated.  After  this  kind  of  artificial  distortion 
process,  the  expected  distortion  d,  the  new  run-length  distributions 
of  black  and  white  runs  p(R]=n)  and  p(Ro=n)  as  well  as  the  entropy 
Hr  (minimum  transmission  rate)  can  also  be  given  in  closed  form. 

The  corresponding  values  of  rates  and  distortions  for  different 

Markov  sources  are  compared  with  the  lower  bound  Rj^(d)  = H(a)  + 

— ^ H(b)  - H(d),  where  H(x)  represents  "-  x-log(x)  - ( 1-x) • log (1-x) " 
a+b 

and  a and  b are  the  conditional  probabilities  p(0/l)  and  p(l/0)  of 
the  binary  source  symbols.  For  many  points  they  are  quite  close 
together.  Computer  simulations  yield  also  the  same  results. 
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DIFFERENTIAL  PULSE-CODE  MODULATION  OF  THE  WIERNER  PROCESS,  Akira 
Hayashi  (Kanazawa  Institute  of  Technology,  Kanazawa,  Japan).  The 
performance  of  differential  pulse-code  modulation  with  the  Wiener 
process  input  is  analyzed.  The  analysis  is  exact:  no  use  is  made 
of  the  concepts  of  slope  overload  error  and  granular  error.  The 
limit  as  n->«>  for  the  characteristic  function  of  error  distribution 
at  time  nT  is  found  in  terms  of  the  step  size  parameter  A and  the 
number  of  levels  N of  quantization.  Uniform  symmetric  quantization 
and  ideal  integration  in  the  feedback  path  are  assumed.  Curves  are 
presented  of  two  kinds  of  mean-squared  error  versus  A or  N,  and  are 
compared  with  the  rate  distortion  function. 


THE  SIMULATION  PROBLEM,  Joseph  Linde  and  Robert  M.  Gray  (Information 
Systems  Laboratories,  Stanford  University,  Stanford  CA  94305).  Let 
X be  a statlonary-ergodic  source,  U an  i.i.d  source  with  finite 
alphabet  A consisting  of  M equally  probable  output  levels  and  let 
p(.,.)  be  a single  letter  distortion  measure.  Consider  the  class  of 
all  time-invariant,  possible  nonlinear,  filters  that  can  be  repre- 
sented as  1 shift  register  consisting  of  N M-level  cells  followed 
by  a fixed  function  F:A^  -*■  R. 

The  simulation  problem  is  to  find,  within  this  class,  the  filter 
which  when  driven  by  U will  yield  an  output  which  is  closest  in  the 
generalized  Ornsteln  or  'p-distance  to  the  given  source  X.  The 
finite  entropy  output  process  is  called  a fake  process. 

Gray  had  shown  that  the  simulation  problem  is  equivalent  to  the  source 
coding  (data  compression)  problem;  in  particular  a good  fake  process 
filter  can  be  used  as  a decoder  in  a source  coding  system,  where  a 
trtllls  or  tree  search  is  used  as  the  encoder,  to  yield  nearly  onti- 
mal  systems. 

We  use  this  approach  to  design  source  coding  systems  for  i.i.d., 
autoregressive  and  moving  average  sources  using  fake  processes  gen- 
erated via  Central  Limit  Theorem  and  inverse  distribution-scrambling 
functions  techniques.  Computer  simulations  are  carried  out  for  the 
case  of  Gaussian  sources  at  the  low  rate  of  1 bit/symbol.  In  this 
case  it  is  shown  that  the  systems  so  designed  outperform  the  optimal 
quantizer  by  0.7  dB  in  the  i.i.d  case  and  outperform  delta-modulation 
and  predictive  quantization  by  1-2  dB  in  the  first  order  autoregres- 
sive and  moving-average  cases. 
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SOME  RELATIONS  BETWEEN  MUTUAL  INFORMATION,  SAMPLE  PATH  PROPERTIES, 
A^JD  SIGNAL  DETECTION,  Charles  R.  Baker  (Department  of  Statistics, 
University  of  North  Carolina,  Chapel  Hill,  NC  27514).  Let  (Sj-)  and 
(Nf),  t in  [0,t],  be  stochastic  processes  having  almost  all  sample 
paths  in  L2L0,TJ.  (N^.)  is  assumed  to  be  zero-mean  and  Gaussian. 
Under  appropriate  assumptions  on  (Sj.) , various  conditions  are  known 
for  finite  mutual  Information  of  signal  and  signal-plus-noise,  non- 
singular  signal  detection,  finite  signal-to-nolse  ratio  (quadratic- 
linear  test  statistic),  and  signal  sample  path  behavior  (in  terms 
of  the  noise  covariance  function).  We  present  results  which  show 
the  connection  between  these  properties,  for  a fixed  signal  process 
(St)  and  a fixed  noise  process  (N^) . Wlien  signal  and  noise  are 
jointly  Gaussian,  the  results  given  here  generalize  previous  results 
due  to  T.S.  Pitcher  and  to  the  author. 


BOUNDS  ON  THE  MUTUAL  INFORMATION  FOR  NONLINFJUl  OBSERVATION  PROCESSES 
WITH  ADDITIVE  WHITE  GAUSSIAN  NOISE,  S.  Arimoto  and  T.  Hashimoto 
(Faculty  of  Engineering  Science,  Osaka  University,  Toyonaka,  Osaka 
560,  Japan).  This  paper  is  concerned  with  the  mutual  information 
between  an  input  signal  x(')  and  an  output  observable  y(")  for 
linear  or  nonlinear  observation  processes  of  type,  y (t)=g(x(t) ,t)  + 
w(t),  where  w(t)  is  an  independent  white  Gaussian  noise.  Lower 
bounds  on  the  mutual  information  obtained  for  discrete-time  processes 
are  expressed  by 


l[x(0,k);  y(o,k)]  = 7 ln[det(W  \w  +P  ))] 

1=0  111 

where  Wj^  are  covariance  matrices  of  the  white  Gaussian  noise  and  Pj^ 
are  certain  non-negative  definite  matrices.  One  of  upper  bounds  is 
described  by 

I ^ 1 

l[x(0,k);  y(0,k)]  = Y ^ ln[det(w"\w^4<)^))]. 

i=0 

In  general,  If  g(x,t)  is  linear  in  x and  x(*)  is  Gaussian, 

then  Qi*Pi  for  all  1.  The  bounds  on  the  mutual  information  for 
i-ont Inuous-t ime  cases  are  described  in  terms  of  Fredholm  determinant 
associated  with  certain  linear  Integral  equations,  which  are  derived 
by  taking  limits  of  discrete-time  processes  as  sampling  period  tends 
to  infinitesimally  small,  provided  that  the  continuous-time  process 
x(.)  is  mean-square  continuous  or  continuous  in  probability. 

Tiie  method  proposed  is  effective  for  not  only  causal  filtering  but 
also  non-causal  filtering.  Some  new  formulae  for  the  mutual  in- 
formation 1(x(t,9);  y(s,t)],  where  s»T«0»t,  are  presented. 


74 


SESSION  C5 


STOCHASTIC  AND  MULTIPLE  WIENER  INTEGRALS  FOR  GAUSSIAN  PROCESSES, 

Stt-el  T.  Huang  (Department  of  Mathematics,  University  of  Cincinnati) 
and  Stamatis  Cambanis  (Department  of  Statistics,  University  of  North 
Carolina  at  Chapel  Hill).  Multiple  Wiener  integrals  and  stochastic 
Integrals  are  defined  for  Gaussian  processes,  extending  the  related 
notions  for  the  Wiener  process.  It  is  shown  that  every  L2-functlonal 
of  a Gaussian  process  admits  an  adapted  stochastic  Integral  repre- 
sentation and  an  orthogonal  series  expansion  in  terms  of  multiple 
Wiener  integrals.  Also  some  results  of  Wiener's  theory  of  nonlinear 
noise  are  generalized  to  noises  other  than  white,  and  the  differential 
formula  for  the  stochastic  integral  is  derived. 


A TIME  PERTURBATION  OF  GAUSSIAN  STOCHASTIC  PROCESSES  AND  SOME 
APPLICATIONS  TO  THE  THEORY  OF  SIGNAL  DETECTION,  A.F.  Gualtierottl 
(Departement  de  mathematlques , Ecole  Polytechnique  Federale,  26, 

Av.  de  Cour,  1007  Lausanne,  Switzerland).  Let  X be  a stochastic 
process  with  square  Integrable  paths.  The  statistician  can  in 
principle  observe  H(X),  the  closure  in  L2  of  the  vector  space  gen- 
erated by  X.  He  is  however  interested  in  L2(X),  that  is,  the  family 
of  "functionals"  of  the  process  (l.e.  the  family  of  functions 
measurable  with  respect  to  the  tribe  generated  by  X and  square  in- 
tegrable). When  X is  Gaussian,  one  has 

L (X)  = I H(X)°", 
n>0 

where  "On"  denotes  the  symmetric  tensor  product  of  n copies  of  H(X). 
Theoretically  then  the  statistician  has  access  to  L2(X).  In  practice 
however  information  about  X is  obtained  through  experimental  observa- 
tions, which  in  turn  lead  to  estimated  parameters.  Decisions  are  then 
based  on  the  latter.  It  is  thus  important  to  evaluate  how  the  above 
relation  between  H(X)  and  L2*lx)  is  affected  when  one  deals  not  really 
with  X,  but  some  perturbation  of  it,  say  Y. 

The  case  of  Y spherically  invariant  has  been  extensively  investi- 
gated. In  the  talk  we  shall  consider  a time  "perturbation"  of  X, 
i.e.  set  Yj.  =•  Xy^j.,  and  study  some  of  its  effects  on  detection  and 
estimation.  The  two  conclusions  that  may  be  derived  are  that  non- 
singularity  of  detection  is  not  unduly  affected,  but  that  the  usual 
procedures  may  introduce  a systematic  bias  in  estimations.  The  re- 
sults are  Indicative  of  what  may  happen,  rather  than  what  really 
happens,  because  calculations  about  Y are  made  supposing  X known. 


ON  THE  RECONSTRUCTION  OF  THE  COV,\RIANCE  OF  STATIONARY  GAUSSIAN 
PROCESSES  OBSERVED  THROUGH  ZERO  MEMORY  NONLINEARITIES,  Stamatis 
Cambanis  (Department  of  Statistics,  University  of  North  Carolina 
at  Chapel  Hill,  Chapel  Hill,  NC  27514)  and  Elias  Masry  (Department 
of  Applied  Physics  and  Information  Science,  University  of  California 
at  San  Diego,  La  Jolla,  CA  92093).  We  consider  the  problem  of 
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reconstructing  the  normiilized  covariance  function  R(t)  of  a zero 
mean  stationary  Gaussian  process  observed  through  a zero  memory 
nonlinearity  f (x) , when  we  know  the  nonlinearity  and  the  correlation 
function  or  the  second  order  distribution  of  the  output  process. 
Three  kinds  of  results  are  established  showing  (1)  how  arbitrary 
covariances  may  be  reconstructed  for  certain  nonlinearities:  in- 
cluded here  are  monotonic  f's,  appropriate  Interval  windows  and  cer- 
tain quite  general  f's;  (ii)  how  certain  covariances  can  be  recon- 
structed for  arbitrary  nonlinear ities : included  here  are  positive 
covariances  (2:0),  covariances  with  rational  spectral  densities, 
and  bandlimited  covariances;  and  (iii)  how  certain  covariances  can 
be  reconstructed  for  certain  nonlinearities:  included  here  are 
covariances  satisfying  certain  rather  weak  conditions,  which  are: 
easily  checked  in  terms  of  the  output  correlation  function,  and 
certain  symmetric  as  well  as  nonsymmetric  f's. 


BOUNDS  ON  ESTIMiMION  ERROR  FOR  GAUSS-POISSON  PROCESSES,  Adrian 
Segal  1 (Department  of  Electrical  Engineering,  Technlon  - Israel 
Institute  of  Technology,  Haifa,  Israel).  Upper  and  lower  bounds 
for  the  estimation  error  of  a linear  Gauss-Markov  process  that 
modulates  the  mean  of  a Gauss-Poisson  process  are  obtained.  The 
upper  bounds  are  derived  by  looking  at  the  best  linear  estimator 
for  the  same  problem.  The  lower  bound  is  obtained  by  using  a 
generalized  Cramer-Rao  bound  developed  by  Bobrovskl  and  Eakai. 


POISSON  S,\MPLING  AND  SPECTRAL  ESTIMATION  OF  CONTINUOUS-TIME 
PROCESSES,  Elias  Masry  (Department  of  Applied  Physics  and  Infor- 
mation Science,  University  of  California  at  San  Diego,  La  Jolla, 

CA  92093).  We  consider  a class  of  spectral  estimates  of  a con- 
tinuous-time stationary  stochastic  process  X = {X(t)  ,-‘^<t<“>}  based 
on  a finite  number  of  observations  {X(tn)}n=i  taken  at  Poisson 

sampling  Inst  ints  (t^).  We  investigate  the  bias  and  covariance  , 

structure  of  the  estimates  and  discuss  the  Influence  of  the  spec- 
tral windows  and  the  sampling  rate  on  the  performance  of  the  es- 
timates. The  estimates  are  shown  to  be  mean-square  consistent  under 
mild  smoothness  condition  on  the  spectral  density.  The  process  X 
need  not  be  bandlimited.  The  results  of  a simulation  study  are 

discussed.  ^ 

I 


76 


PLENARY  SESSION  C 


SOME  RECENT  RESULTS  ON  CHARACTERIZATION  OF  MEASURES  OF  INFORMATION 
RELATED  TO  CODING,  J.D.  Aczel  (Mathematics  Department,  University 
of  Waterloo,  Waterloo,  Canada).  Essential  properties  of  entropies 
and  other  measures  of  Information  are  arrived  at  on  basis  of  the 
noiseless  coding  theorems,  source  entropies  and  Huffman  codes. 

Conversely,  characterization  theorems  are  given,  based  on  these 
properties.  Some  of  the  most  important  are  Shannon's  inequality, 
boundedness  on  an  interval,  subadditivlty , additivity,  branching 
and  expansibility.  Also  entropies  of  mixed  probabilistic  and  non- 
probabllist  Ic  character  and  convex  f-dlvergences  are  dealt  with, 
among  others. 

Some  unsolved  problems  are  stated. 


TEAM  DECISION,  MARKET  SIGNALING,  AND  INFORMATION  THEORY,  Y.C  Ho 
(Harvard  University,  Cambridge,  MA) . This  talk  will  attempt  to 
unify  three  at  least  superficially  separate  fields  of  research: 
team  decision  theory,  market  signaling  in  economics  and  Shannon's 
information  theory. 


INFORMATION  THEORY  IN  PHYSICS,  E.  T,  Jaynes,  Washington  University, 
St.  Louis,  Mo.  (USA).  This  talk  will  survey  the  role  of  Information 
theory  in  physics,  past,  present,  and  future.  There  are  a few 
interesting  confluences  of  mathematical  results  known  to  workers  in 
statistical  mechanics;  for  example,  application  of  Information  theory 
in  predicting  the  course  of  time-dependent  irreversible  processes  led 
to  a formalism  mathematically  isomorphic  with  Norbert  Wiener's 
prediction  theory.  The  same  integral  equations  for  the  optimal 
predictor  appeared,  the  kernel  covariance  functions  now  appearing  as 
thermal  expectation  values  of  products  of  quantum-mechanical 
operators . 
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DECODING  RANDOM  CODES  WITH  AN  OPTIMUM  THRESHOLD,  B.  G.  Dorsch, 
(Instltut  fiir  Nachrichtentechnlk,  DFVLR,  D-8031  Oberpfaf fenhofen, 
Federal  Republic  of  Germany  (BRD)).  For  random  block  codes  a subop- 
timum decoding  rule  is  defined,  which  takes  any  codeword  within  a 
fixed  distance  t from  the  received  word  as  decoding  decision  rather 
than  the  most  probable  one.  It  will  be  shown  by  random  coding 
arguments  for  the  binary  symmetric  channel  that  with  an  optimum 
threshold  t channel  capacity  can  be  approached  by  this  decoding 
rule.  Furthermore  for  rates  close  to  capacity  the  error  exponent 
E(R)  of  this  decoding  principle  is  as  good  as  that  of  maximum 
likelihood  decoding.  Comparisons  are  made  also  with  lower  and  up- 
per bounds  of  the  error  exponent  of  bounded  minimum  distance  de- 
coding BMD  showing  E(R)  is  superior  for  lilgh  rates  and/or  very 
noisy  channels.  Applying  the  results  to  the  additive  white  Gaussian 
noise  channel  with  hard  binary  decisions,  e.g.  for  codes  of  length 
N » 1000  and  rate  yields  an  error  probability  ^ 10“5  with  a signal 
to  noise  ratio  E[j/No  ^ 3.6  dB.  This  is  approximately  2 dB  lower 
than  that  of  BMD  when  the  minimum  distance  approaches  the  Varshamov- 
Gllbert  bound. 


DIGITAL  WHITENING  TECHNIQUES  FOR  IMPROVING  SPREAD  SPECTRUM 
COMMUNICATIONS  PERFORMANCE  IN  THE  PRESENCE  OF  NARROW-BAND  JAMMING 
AND  INTERFERENCE,  Frank  M.  Hsu  and  Arthur  Giordano  (GTE  SYLVANIA, 
Electronic  Systems  Group,  Eastern  Division,  77  "A"  Street, 

Needham  Heights,  MA  02194).  Narrowband  jamming  and  interference 
due  to  other  users  in  spread  spectrum  communications  systems  can  be 
effectively  suppressed  by  using  digital  whitening.  The  received 
spread  spectrum  signal  is  assumed  to  consist  of  the  sum  of  the  SS 
transmitted  signals,  narrowband  interference  and  thermal  receiver 
noise.  Both  the  transmitted  signal  and  the  thermal  receiver  noise 
can  be  assumed  to  have  white  spectral  characteristics.  The  nar- 
rowband interference,  on  the  other  hand,  is  nonwhite  over  the  SS 
signal  band  and  in  the  short  term  is  therefore  coherent.  This 
coherent  component  can  be  predicted  by  use  of  digital  whitening 
techniques  which  can  be  implemented  as  a transversal  filter.  Since 
the  noncoherent  portion  of  the  received  signal  resulting  from  white 
signal  components  is  not  predictable,  the  estimate  formed  can  ac- 
tually be  interpreted  as  an  estimate  of  the  interfering  signal 
component.  The  narrowband  interference  is  then  suppressed  by  sub- 
tracting the  estimate  from  the  received  signal.  As  a result,  an 
impressive  improvement  in  receiver  performance  can  be  obtained. 

In  this  paper,  the  Weiner  and  maximum  entropy  filters  are  used  for 
digital  whitening.  The  merits  of  these  two  filters  in  different 
Jamming  and  signaling  situations  are  compared  and  are  found  to 
exhibit  similar  performance  over  a wide  range  of  input  slgnal-to- 
noise  ratios. 
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RECOVERY  OF  SPREAD  SPECTRUM  CARRIER  FUNCTIONS,  Karl  Heinz  Annecke, 
(Instltut  fur  Elektrlsche  Nachrichtentechnik  der  Rheinisch- 
Westfalischen  Hochschule  Aachen,  D5100  Aachen,  Alte  Maastrichter 
Str.  23,  W.  Germany).  A system  is  described  which  generates  a 
synchronous  spread  spectrum  reference  carrier  for  a coherent  re- 
ceiver from  the  received  signal  by  a decision-directed  process. 

In  a coherent  receiver  it  is  necessary  in  synchronizing  a reference 
carrier  to  the  carrier  of  the  received  signal.  The  synchronization 
is  mostly  done  by  a phase  locked  loop.  In  systems  which  use  a wide- 
band carrier  the  difficulties  to  synchronize  the  reference  carrier 
grow  with  the  bandwidth  of  the  used  carrier.  In  this  paper  the  car- 
rier is  a product  of  subcarriers  c^pCt);  cip(t)  are  bipolar  periodi- 
cally repeated  functions  with  period  Tj^  consisting  of  m^  subpulses 

of  length  t Ic  (t)  ^ c.  (t)  . The  transmitted  Information  is 
o L p i=l 

a bipolar  function  consisting  of  "bits"  of  length  T;  T is  the  period 

of  the  carrier  c (t)  1 T = t m.  . In  the  receiver  each  subcar- 

P L o , ij 


rler  c^^Ct)  is  computed  from  the  received  signal  which 

with  an  estimate  of  the  Information  and  an  estimate  of 

r n 

of  all  the  other  subcarriers 


TT 

k=l 

Lk^i 


is  multiplied 
the  product 


The  systems  properties  are  described  for  the  special  case  n » 1 
and  for  the  common  case  n > 1.  The  theoretically  expected  results 
are  compared  with  computer  simulations  and  hardware  measurements. 


ERROR  PROBABILITY  FOR  AN  INCOHERENT  CHANNEL  WITH  PARTIAL  BAND  OR 
TIME  NOISE  AND  A MISMATCHED  DECODING  STATISTIC,  Steven  Krich, 
(Lincoln  Laboratory,  Massachusetts  Institute  of  Technology, 
Lexington,  MA  02173).  The  bit  error  probability  for  an  incoherent 
additive  Gaussian  noise  channel  with  noise  power  density  permitted 
to  change  for  each  channel  symbol  transmission  has  been  evaluated 
by  Viterbi  and  Jacobs  (1975).  They  assumed  a receiver  which  com- 
putes the  weighted  sum-of-the-squared  matched  filter  envelope  de- 
tectors with  weights  determined  by  an  accurate  estimate  of  the 
channel  noise.  They  showed  that  by  use  of  diversity  (signal  repe- 
tition) or  coding  the  error  probability  could  be  made  to  decrease 
exponentially  with  E(,/Nq  where  E^j  is  the  energy  per  bit  and  Nq  w/Hz 
is  the  average  noise  power  density. 

We  consider  here  the  degradation  in  performance  which  results  from 
an  inability  to  accurately  measure  the  channel  conditions.  For 
Instance,  the  receiver  might  set  the  weights  based  upon  the  measured 
noise  In  a time  Interval  just  prior  to  the  expected  signal.  If 
the  noise  is  quickly  time  varying,  the  weights  will  be  wrong  much 
of  the  time.  In  spite  of  this,  with  diversity  or  coding,  the  error 
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probability  still  decreases  exponentially  in  E(,/No  in  the  presence 
of  partial  time  noise.  At  most  an  additional  3 dB  In  Ejj/Nq  Is  re- 
quired to  compensate  for  the  incorrect  weights.  By  comparison,  the 
error  probability  decreases  linearly  in  Ejj/Nq  if  no  attempt  is  made 
to  measure  the  noise. 

Also  considered  is  a receiver  which  ignores  very  noisy  filter  out- 
puts and  sets  the  remaining  weights  based  upon  an  accurate  measure- 
ment of  the  channel  noise. 


^VDJ AGENT  CHANNEL  INTERFERENCE  IN  A BINARY  BANDPASS  COMMUNICATION 
SYSTEM,  Israel  Korn  (Department  of  Elec.  Eng.,  Faculty  of  Eng. 
Science,  Ben-Gurlon  University,  Beer-Sheva,  Israel)  and  Moshe 
Herzberg  (Faculty  of  Elec.  Eng. , Technion-Israel  Institute  of 
Technology,  Haifa,  Israel).  The  effect  of  adjacent  cliannel  inter- 
ference on  the  probability  of  error  in  a binary  bandpass  communi- 
cation system  with  an  integrating  and  dumping  detector  is  inves- 
tigated. Narrowband  filters  are  assumed  in  the  receiver  of  the  main 
signal  and  transmitters  of  both  main  and  interfering  signals.  Plots 
of  probability  of  error  as  a function  of  signal-to-nolse  ratio  in 
the  main  channel  or  as  a function  of  carrier  frequency  difference 
between  the  main  and  interfering  signals  are  presented,  assuming 
that  the  filters  are  of  the  Butterworth  type.  These  figures  are 
helpful  in  selection  of  minimal  frequency  spacing  of  adjacent 
channels. 


CHA.NNEL  ESTIMATION  AND  DECODING  IN  A MULTIPATH  ENVIRONMENT,  Kenneth  S. 
Schneider  (Network  Analysis  Corp.,  Glen  Cove,  NY)  and  Terrence  P. 
McCarty  (Communications  Satellite  Corp.,  Washington,  D.C.).  In  the 
area  of  mobile  satellite  communications,  one  of  the  serious  limita- 
tions is  the  fading  due  to  specular  multipath.  In  an  attempt  to 
mitigate  against  its  effect,  large  aperture  tracking  antennas  have 
been  used  on  ships.  However,  such  antennas  are  impractical  for  most 
land  vehicles  and  aircraft  so  an  alternative  is  sought.  Recent  work 
by  Schneider  and  McCarty  has  shown  that  a signal  processing  technique 
using  the  intersymbol  Interference  model  proposed  by  Forney  and  the 
Vlterbl  decoder  can  provide  adequate  communications  performance  in 
the  presence  of  specular  multipath. 

A limitation  of  this  technique  was  that  it  required  that  estimates 
of  the  specular  multipath  delay  and  reflections  coefficient  be  obtained 
for  the  matched  filter.  This  is  often  a difficult  set  of  estimates 
to  obtain  so  that  a robust  scheme  was  proposed.  The  robust  scheme 
uses  a known  matched  filter  and  develops  an  equivalent  finite  state 
machine  model.  For  this  case  the  tap  gains  were  to  be  estimated. 

Using  reliable  tap  gains,  it  was  shown  that  the  performance  of  this 
robust  scheme  was  similar  to  that  of  the  optimum  scheme  with  perfect 
channel  knowledge. 
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In  this  paper  the  authors  develop  a technique  to  obtain  the  desired 
channel  tap  gains.  This  is  done  by  using  a Kalman  filter  with  a 
training  sequence.  Schemes  of  this  type  have  been  proposed  by 
Lawrence  and  Kaufman,  and  by  Godard.  We  have  built  upon  the  latter 
technique  and  extended  it  to  the  vector  measurement  channel  used  by 
the  robust  scheme.  The  paper  develops  performance  results  and  dis- 
cusses them  in  terms  of  optimal  sets  of  training  sequences. 


STOCHASTIC  CHANNELS  AS  GENERALIZED  COMMUNICATION  NETWORKS,  David 
Middleton  (127  East  91st  Street,  New  York,  NY  10028)  and  J.  Raymond 
Breton  (Naval  Underwater  Systems  Center,  New  London,  CT  06320). 
Conceptually  it  is  perhaps  well  known  that  physical  channels  through 
which  information-bearing  signals  are  transmitted  can,  in  a rather 
loose  sense,  be  regarded  as  space-time  filters  or  generalized  networks, 
analogous  to  the  temporal-only  filters  or  networks  of  conventional 
circuit  and  control  theory.  What  has  not,  however,  been  constructed 
quantitatively  is  the  analogous  theory  of  such  distributed  filters, 
or  networks,  taking  into  particular  account  their  capabilities 
for  signal  degradation  and  noise  generation,  specifically  in  terms 
of  the  controlling  physics  of  the  channel  and  the  random  and  deter- 
ministic inhomogeneities  which  are  the  critical  factors  influencing 
Information  transmission. 

! New  results  Include  various  operational  and  perturbation-theoretical 

solutions  for  the  generalized  network  response  functions;  causality 
conditions  which  are  the  space-time  equivalent  of  the  Paley-Weiner 
conditions  of  (linear)  temporal  filters;  control  theory  formulations 
which  provide  general  algorithms  for  simulation  and  computational 
evaluation;  various  preliminary  results  for  space-time  stability  con- 
ditions; procedures  for  calculating  any  desired  channel  statistics 
in  terms  of  an  appropriate  equivalent  deterministic  channel  (or  dis- 
tributed network).  The  approach  is  applicable  to  both  linear  and 
nonlinear  systems,  and  is  interdisciplinary,  involving  such  disci- 
plines as  control  theory,  information  theory,  stochastic  processes, 
the  physics  of  propagation  and  scattering,  and  computational  methods 
and  techniques.  It  is  illustrated  here  in  detail  with  explicit 
results  pertinent  to  electromagnetic  and  acoustical  transmission. 


PROLATE  SPHEROIDAI.  WAVE  FUNCTIONS  - THE  DISCRETE  CASE,  David  Sleplan 
(Bell  Laboratories  and  University  of  Hawaii).  A discrete  time  series 
has  associated  with  it  an  amplitude  spectrum  which  is  a periodic  func- 
tion of  frequency.  This  paper  investigates  the  extent  to  which  a 
time  series  can  be  concentrated  on  a finite  index  set  and  also  have 
its  spectrum  concentrated  on  a sub-interval  of  the  fundamental 
period  of  the  spectrum.  Key  to  the  analysis  are  certain  sequences, 
called  discrete  prolate  spheroidal  sequences,  and  certain  functions 
of  frequency  called  discrete  prolate  spheroidal  functions.  Their 
mathematical  properties  are  investigated  in  great  detail  and  many 
applications  to  signal  analysis  are  pointed  out. 


81 


SESSION  D2 


INTRINSIC  DIMENSIONALITY  SPATIAL-TEMPORAL  ARRAY  PROCESSING, 

Salvatore  D.  Morgera  (Raytheon  Company,  Submarine  Signal  Division,, 
Systems  Laboratory,  Portsmouth,  RI  02871).  This  work  is  concerned 
with  efficient  spatial-temporal  array  processing  for  the  purpose  of 
detecting  a t.-irget  waveform  in  an  unknown  Interference  noise  environ- 
ment. A new  and  rather  general  characterization  of  the  interference 
noise  field  spat lal-temporal  covariance  structure  is  presented. 

The  covariance  model  is  felt  to  be  applicable  to  the  acoustic,  radar, 
and  seismic  areas  and  conveys  an  appreciation  of  the  fact  that  the 
intrinsic  dimensionality,  i.e.,  the  number  of  linearly  Independent 
spatial-temporal  parameters,  of  the  Interference  noise  field  is 
generally  much  smaller  than  the  physical  or  "full"  dimensionality 
of  commonly  employed  array  processors.  The  structure  and  perfor- 
mance of  a reduced  dimensionality  array  processor  is  investigated, 
with  the  performance  compared  to  both  a full  dimensionality  spatial- 
temporal  processor  and  to  just  a temporal  processor.  It  is  shown 
that  the  performance  of  the  reduced  dimensionality  spatial-temporal 
processor  can  be  highly  competitive  with  that  of  the  full  dimen- 
sionality processor  if  (all  but  one  of)  the  spatial  "look"  direc- 
tions are  chosen  as  the  principal  components  of  the  interference 
noise  vector  stochastic  process  corresponding  to  the  largest  values 
of  interference  noise-target  waveform  spatial-temporal  correlation. 
Since  the  principal  components  are  not  a priori  known,  a scheme  is 
presented  for  estimating  the  "best"  set  of  spatial  directions.  A- 
part  from  the  obvious  advantages  of  the  reduced  dimensionality  pro- 
cessor in  the  way  of  minimum  throughput  and  memory  demands,  the 
reduced  dimension  permits  maximum  convergence  rate  and,  consequently, 
utmost  sensitivity  to  a quasl-statlonary  Interference  noise  environ- 
ment for  an  adaptive  implementation. 


NONLINEAR  FEATURE  EXTRACTION  WITH  A CRITERION  OF  A GENERAL  FORM, 
Kelnosuke  Fukunaga  and  Robert  D.  Short  (School  of  Electrical  Engi- 
neering, Purdue  University,  West  Lafayette,  IN  A7907).  In  this 
paper  the  optimal  nonlinear  feature  extraction  for  a criterion 

. ,K^)  (where  the 
second  order  moments) 
o be  a parametric 
function  of  the  conditional  densities.  By  imposing  a further 
restriction  on  the  functional  dependence  of  f on  the  K 's,  the  op- 
timal mapping  takes  on  an  intuitively  pleasing  function  of  the 
posteriori  probabilities.  Next,  the  optimum  feature  mapping  is 
restricted  to  a finite  dimensional  subspace.  The  resulting  optimum 
mapping  is  a linear  combination  of  the  projections  of  the  posteriori 
probabilities  onto  the  subspace.  The  problem  of  finding  the  best 
single  feature  selection  is  discussed  in  this  framework.  Finally, 
several  examples  are  also  discussed. 


function  of  the  general  form  f(D.,...,D  , K^^,.. 
D.'s  and  the  K 's  are  the  conditional  first  and 
lij  considered.-^  The  optimal  solution  is  found  t 
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AN  ALGORITHM  FOR  OPTIMAL  NONLINEAR  STRUCTURE  PRESERVING  FEATURE 
EXTRACTION,  Scott  A.  Starks  and  Rul  J.P.  de  Flgueiredo  (Rice  Uni- 
versity, Houston,  TX  77001).  This  talk  presents  a new  approach  to 
nonlinear  structure  preserving  feature  extraction.  This  method  is 
based  on  certain  graph  theoretical  considerations  (such  as  the 
minimal  spanning  tree,  edge  Inconsistency,  and  diameter  edges)  and 
topographical  considerations  (such  as  Interpolnt  distance  measures). 
After  introduction  of  the  subject  matter  and  appropriate  background 
material,  the  algorithm  is  formulated. 

Numerical  results  from  the  application  of  this  algorithm  to  various 
test  data  sets  are  presented.  Evaluation  of  these  test  results  are 
quite  encouraging. 


RECURSIVE  FACTOR  ANALYSIS  METHODS  IN  FEATURE  EXTRACTION  PROBLEMS, 

L.  Kurz  and  C.  S.  Yoon  (Polytechnic  Institute  of  New  York,  333  Jay 
Street,  Brooklyn,  NY  11201).  The  problems  of  data  reduction,  fea- 
ture extraction  and  pattern  recognition  using  factor  analysis  tech- 
niques were  studied  previously.  The  basic  problem  considered  in 
this  paper  is  to  find  the  estimates  of  the  factor  loading  matrix  A, 
and  the  unique  matrix,  based  on  partitioned  observations  with 

the  assumption  that  the  underlying  statistical  parameters  undergo 
slow  change.  Tills  assumption  corresponds  to  many  useful  data  sets 
such  as  sleep-stage  encephalograms,  earth  resources  satellite  data, 
etc.  The  main  approach  is  to  obtain  the  desired  estimates  from  the 
partitioned  data  by  maximum  likelihood  estimation  of  in  conjunction 


83 


SESSION  D2 


with  estimation  of  A by  a stochastic  approximation  procedure.  The 
approach  used  in  this  paper  yields  an  efficient  computer-oriented 
algorithmic  approach  to  the  feature  extraction  problem. 


TE.XTURE  MODELING  USING  STOCHASTIC  TREE  LANGUAGES,  S.Y.  Lu  and  K.S. 
Fu  (School  of  Electrical  Engineering,  Purdue  University,  West 
Lafayette,  IN  47907).  In  the  proposed  stochastic  model,  a texture 
pat-  ern  is  divided  into  fixed-size  windows.  A stochastic  tree 
grammar  (STG)  is  then  used  to  characterize  windowed  patterns  of  the 
same  class.  The  construction  of  an  STG  from  a training  texture 
pattern  is  as  follows:  A single  pixel  or  a window  of  n x n pixels 
with  a relatively  uniform  gray  level  is  chosen  to  be  the  basic  ele- 
ment of  the  texture  pattern.  Gray  levels  of  basic  elements  are 
treated  as  pattern  primitives.  Then,  a windowed  pattern  is  trans- 
formed into  a tree  representation.  Each  basic  element  in  the  win- 
dow corresponds  to  a node  in  the  tree  representation;  hence,  a 
pattern  primitive  is  represented  by  a label  assigned  to  its  corres- 
ponding node.  Consequently,  we  have  a set  of  trees  obtained  from 
the  training  patterns.  An  STG  can  then  be  contracted  from  the  set 
of  trees  to  model  the  class  of  training  patterns. 

Texture  synthesis,  generation  and  discrimination  using  the  proposed 
stochastic  model  are  reported. 


BEST  FIRST  PARSING  OF  NOISY  WAVEFORMS,  Laveen  N.  Kanal  and  George  C. 
Stockman  (Laboratory  for  Pattern  Analysis,  Department  of  Computer 
Science,  University  of  Maryland,  College  Park,  MD  20742).  This 
paper  presents  a paradigm  for  linguistic  analysis  of  noisy  waveforms, 
which  views  analysis  as  simultaneous  search  of  paths  in  the  model 
space  and  the  data  space.  A set  of  possible  paths,  commonly  called 
segmentations,  are  explored  by  the  primitive  feature  extractor. 

Each  data  path  is  of  the  form  = d^  i(Pi^  ]^).."d^  where 

di  ; e Vq-,  the  terminal  vocabulary  o^  the ’grammar ’and  Pl,j  e [0,1] 
is  the  corresponding  confidence  or  probability  of  extraction.  The 
purpose  of  the  grammar  model  is  to  accept  only  those  segmentations 
that  have  correct  syntactic  structure,  and  the  goal  of  analysis  is 
to  develop  all  correct  Interpretations  if  necessary  and  in  best- 
first  order.  A convenient  evaluation  for  a correct  interpretation 
is  f(6i)  - min  {Pj^,j:j  “ l,nj^}. 

All  possible  paths  through  a context  free  grammar  G = <S,VN,V.f,P> 
can  be  explored  via  state  space  search  by  starting  with  the  initial 
symbol  S and  canonically  generating  all  sentential  forms.  If  s ■ By 
is  any  sentential  form  generated,  where  B " t,...tk  is  a string  of 
terminals^  then  the  degree  of  match  between  s and  the  data  is  de- 
fined as  f(s)  “ min  (Pj^J  ” l,k}  where  B is  the  Initial  segment  of 
some  segmentation  6i  - tj^  (P^)  . . . tk(Pk)  di,k+l(Pi.  k+l)'-.di,ni 
(Pl,nj^).  If  P is  not  the  initial  segment  of  any  segmentation,  then 
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f(s)  » 0 and  the  search  path  is  dead  at  state  s.  If  s = 3 = i5i  for 
some  segmentation,  then  3 is  a final  state  of  the  search  and  a correct 
interpretation  of  the  data  having  confidence  f(3)  = min  {Pi, j : j=l  ,ni} . 
Since  f(s)  > f(3)  for  any  sentence  3 derivable  from  sentential  form 
s,  a best-first  search  of  the  space  of  sentential  forms  is  an  ad- 
missible algorithm  for  producing  the  highest  confidence  interpreta- 
tion of  the  data.  Because  the  search  paradigm  is  so  simple,  it  is 
easily  extended  to  Include  bottom-up  and  non-left-right  syntactic 
operators  and  extraction  of  primitives  under  syntax  control. 


SYNTACTIC  SIGNAL  PROCESSING,  Francois  Le  Chevalier,  Gerard  Boblllot 
and  Ce^cile  Fugier-Garrel  (Office  National  d'Etudes  et  de  Recherches 
Aerospatiales  (ONERA)  92320  Chatillon,  France) . A new  representation 
of  signals  as  a string  of  simple  elementary  signals  whose  succession 
obeys  syntactic  rules  is  presented;  the  extraction  of  such  signals 
can  then  be  performed  in  real  time  by  using  an  original  syntactic 
decoding  method,  DESCOA,  effective  for  a wide  variety  of  type  0 
languages. 

Extensions  of  this  method  lead  to  a true  syntactic  signal  processing, 
including  adaptive  syntactic  processing  and  syntactic  filtering. 
Simulation  results  are  shown  for  three  examples:  radar  target  iden- 
tification, syntactic  processing  of  frequency-modulated  signals, 
and  adaptive  syntactic  antenna  processing. 


VISUAL  PERCEPTION,  INVARIANTS,  NEURAL  NETS,  H.D.  Block,  D.C.  Lewis, 
and  R.H.  Rand  (Department  of  Theoretical  and  Applied  Mechanics, 

Cornell  University,  Ithaca,  NY).  One  problem  in  the  modelling  of 
perception  is  to  recognize  that  the  same  object  is  being  observed, 
even  when  the  visual  representation  of  that  object  on  the  retina 
is  subjected  to  various  transformations.  That  is,  we  seek  an  out- 
put which  is  invariant  under  a class  of  transformations  on  the  in- 
puts. 

We  first  formulate  mathematically  the  classes  of  transformations  to 
which  the  patterns  will  be  subjected;  e.g.  translation,  rotation, 
dilation,  projection,  irregular  illumination,  motion,  distortion,  etc. 
Next  we  onsider  various  processing  operators  applied  to  the  patterns; 
e.g.  autocorrelation,  Fourier  transform,  Laplacian,  gradient,  convo- 
lution and  more  general  operators.  The  successive  application  of 
the  pattern  transformations  and  processing  operations  have  an  alge- 
braic structure  which  we  are  studying,  with  particular  Interest  in 
the  invariants ; l.e.  those  operators  which  are  unchanged  by  a class 
of  transformations  applied  to  the  visual  patterns.  Finally  we  pre- 
sent neural  networks  which  implement  the  various  processing  operators, 
and  we  consider  their  biological  plausibility. 
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BOUNDS  ON  THE  DELAY  COMPLEXITY  OF  ERROR  CORRECTING  CODES,  Yeshoshua 
Imber  and  John  E.  Savage  (Program  In  Computer  Science,  Brovm  Uni- 
versity, Providence  RI  02912).  The  time  to  decode  is  Important  in 
high-speed  data  communications  and  in  other  applications  including 
error-correction  in  computer  memories.  The  delay  complexity  of  de- 
coders, which  is  a measure  of  decoding  time,  is  examined  and  lower 
bounds  are  derived  in  terms  of  the  rate  and  error-correction  capa- 
bility of  the  code  used  and  the  probability  of  error  provided. 
Decoders  and  codes  that  achieve  many  of  the  lower  bounds  are  des  - 
cribed.  In  addition  tradeoffs  between  code  rate  and  decoding  time 
are  studied  for  three  types  of  single  error  correcting  codes. 


TIME  TO  FIRST  ERROR  FOR  AN  INTERLEAVED  CODE  ON  A BURST-NOISE  CHANNEL, 
Robert  A.  Rutledge  (IBM  Corporation,  D18  - 707-2,  P.O.  Box  390, 
Poughkeepsie,  NY  12602).  A bit-interleaved  random  error  correcting 
code  on  a burst-noise  channel  is  considered.  The  channel  noise 
distribution  is  assumed  to  follow  a simple  Markovian  model.  The 
probability  that  a codeword  is  uncorrectable  (PWE)  was  obtained  in 
a previous  paper.  The  present  work  gives  tight  bounds  on  the  mean 
time  to  the  first  uncorrectable  codeword  (MTBF) . The  effect  of  the 
degree  of  interleaving  on  each  of  these  parameters  is  illustrated 
with  an  example.  It  is  shown  that  (for  this  example)  a small  degree 
of  interleaving  will  increase  MTBF,  but  also  Increase  PWE. 

A method  for  extending  these  results  to  the  more  interesting  case 
of  a block- inter  leaved , burst  error  correcting  code  is  indicated. 


LONG  BLOCK  CODES  CAN  OFFER  GOOD  PERFORMANCE,  David  Chase  and  H.  David 
Goldfein  (CNR  Inc.,  220  Reservoir  Street,  Needham,  MA  02194).  Binary 
algebraic  decoders  can  be  used  directly  with  channel  measurement 
(soft  decision)  information  when  algorithms  within  the  class  ori- 
ginally proposed  by  Chase  are  utilized.  These  algorithms  are  based 
on  iterating  a binary  decoder  by  inverting  received  binary  digits 
which  are  likely  to  be  Incorrect  and  selecting  the  most  likely 
error  pattern  when  more  than  one  solution  is  found.  In  this  paper 
a technique  is  Introduced  for  using  short  block  codes  to  reduce  the 
number  of  iterations  required  to  achieve  a given  level  of  perfor- 
mance when  decoding  long  block  codes  with  channel  measurement  in- 
formation. The  effectiveness  of  this  approach  is  demonstrated  by 
decoding  an  (N,K;D)  » (128,64;22)  BCH  code  with  the  use  of  a (23, 
12;7)  Golay  code  for  perturbing  the  raw  data  which  feeds  the  binary 
decoder.  The  simulated  performance  indicates  that  bit  errors  below 
10”5  can  be  achieved  at  a slgnal-to-nolse  ratio  per  information  bit, 
Eb/Ng,  as  low  as  3.5  dB  over  an  additive  Gaussian  noise  channel. 

This  is  believed  to  represent  the  present  state  of  the  art  for  de- 
coding rate-*5  block  or  convolutional  codes. 
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IMPROVEMENTS  IN  BLOCK-RETRANSMISSION  SCHEMES,  John  J.  Metzner 
(Electrical  and  Computer  Engineering  Department,  Wayne  State 
University,  Detroit,  Ml  A8202) . Consider  the  case  where  an  (N,K) 
block  encoded  word  cannot  be  decoded  reliably  and  a second  block 
of  N redundant  digits  is  sent  to  allow  the  receiver  to  make  a new 
try  based  on  the  combined  information  received.  (It  is  assumed 
that  a likelihood  ratio  or  at  least  a "soft"  decision  is  retained 
for  each  digit.)  Two  classes  of  schemes  are  proposed  and  analyzed 
which  give  significantly  better  performance  than  is  obtained  by 
sending  a repeat  of  the  first  block,  yet  do  not  require  excessive 
decoding  complexity. 

One  approach  is  to  take  small  sub-blocks  of  the  original  N-dlglt 
code  as  the  data  digits  of  a short  rate  one-half  code.  For  example, 
for  a sub-block  of  4 binary  digits  the  second  sending  can  complete 
an  (8,4)  Reed-Muller  code  with  a minimum  distance  of  4.  (With 
retransmission  of  the  four  digits,  the  minimum  distance  (original 
and  repeat)  would  be  2).  The  likelihood  ratio  information  can  be 
used  to  make  "hard"  (binary)  decisions  for  the  four  digits  in  the 
subcode  (based  on  the  (8,4)  code  structure.)  To  allow  for  the 
case  where  the  first  sending  is  garbled,  the  encoding  should  permit 
the  data  to  be  extractable  from  the  second  sending  alone. 

The  other  approach  is  to  treat  the  first  sending  as  the  data  digits 
of  a systematic  convolutional  code  of  short  constraint  length.  The 
likelihood  ratios  retained  from  the  first  sending  can  be  used  in 
Viterbl-decoding  type  procedures,  with  the  original  (N,K)  code  used 
for  a final  check.  Constraint  lengths  as  short  as  two  or  three 
digits  can  yield  significant  Improvement  over  retransmission  in 
some  situations. 


ANALYSIS  OF  MERGING  IN  THE  BIT-BY-BIT  DIRECTION/DECODING  ALGORITHM, 
B.D.  Fritchman  and  J.C.  Mlxsell  (Department  of  Electrical  Engineering, 
Building  19,  Lehigh  University,  Bethlehem,  PA  18015).  Even  though 
the  Viterbl  algorithm  is  not  strictly  sequential,  part  of  its 
value  lies  in  the  fact  that  under  most  conditions,  the  computational 
process  for  the  decision  on  the  k^h  symbol  truncates  (merges)  at 
some  step  k+D,  D>L-1.  The  algorithm  truncates  quickly  when 
the  SNR  is  large,  and  more  slowly  when  it  is  small. 

We  show  that  the  sequential  blt-by-blt  algorithm  also  exhibits  this 
property  of  merges,  but  its  characteristics  are  the  reverse  of  the 
Viterbl  algorithm.  If  the  algorithm  has  not  merged  at  the  con- 
straint length,  merging  beyond  the  constraint  length  becomes  slower 
as  the  SNR  Increases. 

The  quality  of  the  decision  is  investigated  as  D Increases  by  de- 
creasing the  decision  for  the  k^h  symbol  and  allowing  the  delay 
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constraint  D to  increase.  It  is  shown  that  a value  D*  exits  such 
that  for  values  D > D* , no  change  in  the  decision  error  can  occur. 

A gec.jietric  signal  space  description  of  the  conditions  for  a merge 
at  any  step  is  given.  The  conditions  presented  apply  to  the  de- 
tection of  signals  in  the  presence  of  intersymbol  Interference,  to 
the  decoding  of  convolutionally  encoded  information,  and  to  the 
joint  detection/decoding  problem. 


IMPLEMENTATION  OF  DECODERS  FOR  BLOCK  CODES,  E.R.  Berlekamp  (Univer- 
sity of  California,  Berkeley,  CA) . This  talk  will  describe  several 
decoders  which  the  author  recently  designed. 

A decoder  for  the  (31,15)  Reed-Solomon  code  had  been  implemented 
in  standard  t2l  technology,  using  a total  of  78  16-pin  Integrated 
circuit  packages.  This  decoder  runs  on  a synchronous  clock  whose 
cycle  time  can  be  as  short  as  120  nanoseconds.  The  decoder  can 
correct  any  pattern  of  t errors  and  s erasures  for  which  2t+s  ^16 
in  no  more  than  3559  clock  cycles.  This  high  speed  is  achieved  by 
a novel  architecture.  No  microprocessors  are  used. 

Extrapolations  of  this  performance  to  longer  block  lengths  and  lar- 
ger alphabet  sizes  Indicate  that  on  quieter  channels  a much  higher 
throughput  can  be  achieved  by  using  higher  rate  RS  codes  of  much 
longer  block  lengths  with  only  a small  increase  in  hardware  com- 
plexity. 

We  have  also  designed  high-speed  decoders  for  long  binary  codes 
using  soft  decisions. 

Traditional  analyses  of  the  performance  of  Reed-Solomon  decoders 
in  the  presence  of  both  errors  and  erasures  have  assumed  that  de- 
coding failure  occurs  only  if  there  is  no  codeword  within  the  ap- 
propriate distance  of  the  received  word.  In  applications  where  de- 
coding error  is  much  more  costly  than  decoding  failure,  performance 
can  be  Improved  by  introducing  the  notion  of  "false  erasures". 

These  are  characters  which  are  correct  even  though  the  demodulator 
erased  them  (presumably  because  one  or  more  wrong  signals,  as  well 
as  correct  signal,  scored  above  some  threshold).  In  a typical 
Gaussian  noise  environment,  about  20  or  25  percent  of  the  "erased" 
characters  may  actually  be  false  erasures,  and  this  fraction  can 
be  increased  by  modifying  the  demodulator's  thresholds.  Since  this 
^-i.tlon  of  false  erasures  is  significantly  larger  than  q-1,  the 
false  erasures  can  be  used  to  provide  an  increased  capability  of 
detecting  decoding  errors.  This  capability  is  especially  helpful 
on  blocks  which  contain  many  erasures.  For  example,  since  the 
(31,15)  Reed-Solomon  code  is  maximum  distance  separable,  any  word 
containing  16  erasures  can  be  decoded  to  a codeword,  even  though 
this  codeword  is  correct  only  if  the  block  contains  no  errors. 
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However,  after  the  decoder  has  "corrected"  the  16  erasures,  a post- 
erior check,  of  its  accuracy  can  be  obtained  by  counting  the  purported 
number  of  "false  erasures",  e.g.,  the  number  of  erasures  in  which  the 
purported  correction  consisted  of  no  change.  If  the  block  .indeed 
contained  no  character  errors,  then  the  expected  number  of  false 
erasures  is  25  per  cent,  or  4 characters,  but  if  the  block  con- 
tained one  or  more  errors,  and  the  decoding  is  therefore  incorrect, 
then  the  expected  number  of  false  erasures  is  1/32,  or  about  1/2 
of  a character. 

When  additional  processing  time  is  available,  the  decoder  might  also 
use  the  false  erasures  to  correct  many  error  patterns  with  2t+s  > 15. 


CODING  FOR  NONPROBABILISTIC.  DISCRETE  CHANNELS,  William  L.  Root 
(The  Universitv  of  Michigan,  Ann  Arbor,  MI  48109).  Coding  problems 
are  considered  for  discrete  channels  with  finite  alphabets.  The 
mappings  from  input  to  output  are  deterministic,  but  are  & priori 
unknown  to  sender  and  receiver  and  may  change  from  time  to  time. 
These  mappings  do,  however,  belong  to  a class  of  mappings  known  to 
the  communicators.  With  error-free  decoding  the  criterion  of  sat- 
isfactory transmission,  maximum  rates  are  estimated,  or  in  a few 
instances  calculated,  for  various  classes  of  channel  mappings. 

Most  of  the  channels  are  memoryless;  one  is  a special  channel  with 
finite  memory. 


CHAIN  CODING  OF  TABULAR  DATA  IN  NOISY  ENVIRONMENTS,  L.  Kurz  and  C. 
Mohwinkel  (Polytechnic  Institute  of  New  York,  333  Jay  Street, 
Brooklyn,  NY  11201).  . In  this  paper,  some  methods  for  reducing  tabu- 
lar data  computed  by  noise  into  a string  language  by  means  of  chain 
encoding  are  presented.  The  methods  of  detection  of  chain  angle, 
chain  construction,  and  the  use  of  positional  information  of  the 
input  data  for  checks  of  position  and  accuracy  of  any  individual 
chain  element  guarantee  accurate  description  of  the  object  by  the 
chain  codes.  The  data  reduction  methods  presented  in  this  paper 
are  useful  in  Increasing  speed  and  efficiency  of  tabular  (pictorial) 
data  transmission  and  in  reducing  expensive  storage  and  processing 
time  of  the  received  encoded  image.  In  addition,  chain  coded  data 
has  certain  Indigenous  properties  which  make  it  useful  in  certain 
types  of  recognition  and  correlation  problems. 
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ROBUST  RATE  DISTORTION  THEORY,  Dimitri  ICazakos  (State  University  of 
New  York,  Buffalo)  and  T.  Papantoni  - Kazakos  (Bell  Laboratories  and 
Rice  University).  The  practical  utility  of  Rate-distortion  theory 
is  hampered  by  the  fact  that  exact  knowledge  of  source  statistics 
is  necessary  in  order  to  seek  the  optimal  source  codes.  Recently, 
several  researchers  developed  source  coding  techniques  for  unknown 
source  statistics.  However,  an  unduly  high  complexity  of  implemen- 
tation limits  their  practicality. 

In  the  present  paper  we  consider  the  adaptation  of  Rate-distortion 
theory  to  the  situation  of  incompletely  known  source  statistics. 

We  assume  that  the  source  statistics  are  members  of  a convex  family 
and  we  seek  the  worst  source  within  the  family.  The  explicit  calcu- 
lation for  some  examples  is  carried  out. 


NEW  RESULTS  ON  CODING  OF  STATIONARY  NONERGODIC  SOURCES,  Lee  D. 
Davisson  and  Alberto  Leon-Garcla  (University  of  Maryland,  College 
Park,  MD)  and  David  L.  Neuhoff  (University  of  Michigan,  Ann  Arbor, 
MI).  We  present  two  results  on  the  coding  of  stationary  nonergodlc 
sources.  The  first  is  a converse  information  transmission  theorem. 
The  least  achievable  average  distortion  in  transmitting  the  output 
of  a stationary  nonergodlc  source  through  a channel  of  capacity  C 
using  block  stationary  encoder/decoder  pairs  is  given  by  6(C).  In 
general,  6(C)>D(C),  the  distortion-rate  function.  Gray  and  Davisson 
have  shown  that  6(R)  is  also  equal  to  the  least  average  distortion 
achievable  by  the  fixed  rate  coding  of  stationary  nonergodlc  sources. 
Our  second  result  is  a source  coding  theorem  stating  that  there  ex- 
ist variable  rate  codes  which  achieve  R(D),  the  rate-distortion 
bound,  and  hence  also  achieve  D(R) . 


VARIABLE-RATE  UNIVERSAL  BLOCK  SOURCE  CODING  SUBJECT  TO  A FIDELITY 
CONSTRAINT,  K.M.  Mackenthun,  Jr.,  and  M.B.  Pursley  (Coordinated 
Science  Laboratory,  University  of  Illinois,  Urbana,  IL  61801). 
Previous  results  on  universal  variable-rate  source  coding  subject 
to  a fidelity  constraint  are  given  in  terms  of  a probability  measure 
W on  a collection  A of  stationary  ergodic  sources.  Loosely  speaking, 
these  results  imply  that  there  exist  variable-rate  codes  which  yield 
average  distortion  D at  an  optimum  rate  ^0(0)  for  all  sources  6 in 
some  subset  A’  c A.  For  arbitrary  e > 0,  A'  can  be  chosen  to  satisfy 
W(A  - A')  <e. 

New  results  are  presented  which  do  not  require  a probability  measure 
on  A.  In  particular,  conditions  are  given  under  which  there  exist 
variable-rate  codes  whose  rate-distortion  performance  converges  to 
Rg(D)  for  each  source  0 in  the  collection  A.  TVo  such  forms  of 
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universal  coding  are  considered.  For  the  weaker  of  these  two 
(weakly-universal)  the  convergence  Is  polntwlse,  and  for  the  stronger 
(strongly-unlversal)  the  convergence  Is  uniform  on  A. 

Existence  of  weakly-universal  variable-rate  codes  is  established 
for  a collection  of  stationary  ergodic  sources  under  very  general 
conditions.  One  set  of  conditions  is  as  follows:  (1)  the  source 
and  reproduction  alphabets  are  separable  metric  spaces,  (2)  the 
distortion  measure  is  a nonnegative,  nondecreasing,  continuous 
function  of  the  metric,  (3)  there  exists  a reproduction  letter  giving 
finite  average  distortion  for  all  sources  in  the  collection,  and 
(4)  sup{Rg(D):  0 e A}  < “.  Other  sets  of  conditions  are  given 
which,  for  instance,  replace  (4)  by  conditions  on  the  memory  of 
sources  in  the  class.  Examples  of  collections  of  sources  which  are 
covered  by  these  other  sets  of  conditions  but  do  not  satisfy  (4) 
Include  the  collection  of  all  stationary  ergodic  Gaussian  sources 
with  rational  spectral  density. 

Existence  of  strongly-unlversal  variable-rate  codes  is  established 
for  collections  of  finite  alphabet  memoryless  sources  and  certain 
types  of  Markov  sources.  For  example,  it  is  shown  that  universal 
codes  exist  for  the  collection  of  all  binary  first-order  Markov 
sources,  but  they  do  not  exist  for  the  class  of  all  finite-order 
Markov  sources. 


COMPRESSION  OF  INDIVIDUAL  SEQUENCES  VIA  VARIABLE-RATE  CODING,  Jacob 
Ziv  (Bell  Laboratories,  Murray  Hill,  New  Jersey  07974)  and  Abraham 
Lempel  (Sperry  Research  Center,  Sudbury,  MA  01776).  Compressibility 
of  individual  sequences  by  the  class  of  generalized  finite-state, 
information-lossless  encoders  is  Investigated.  These  encoders  can 
operate  in  a variable-rate  mode  as  well  as  a fixed-rate  one,  and 
they  allow  for  any  finite-state  scheme  of  variable-length-to-variable- 
length  coding.  For  every  individual.  Infinite  sequence  x we  define 
a quantity  p(x),  called  the  compressibility  of  x,  which  is  shown  to 
be  the  asymptotically  attainable  lower  bound  on  the  compression-ratio 
which  can  be  achieved  for  x by  any  finite-state  encoder.  This  is 
demonstrated  by  means  of  a constructive  coding  theorem  and  its  con- 
verse which,  apart  from  their  asymptotic  significance,  also  provide 
useful  performance  criteria  for  finite,  and  practical,  data-compres- 
slon  tasks.  The  proposed  concept  of  compressibility  is  also  shown 
to  play  a role  analogous  to  that  of  entropy  in  classical  information 
theory,  where  one  deals  with  probabilistic  ensembles  of  sequences 
rather  than  with  individual  sequences. 


CODING  THEOREMS  FOR  INDIVIDUAL  SEQUENCES,  J.  Ziv  (Bell  Telephone 
Laboratories,  Inc.,  Murray  Hill,  NJ  07974).  A quantity  called  the 
finite-state  complexity  is  assigned  to  every  infinite  sequence  of 
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elements  drawn  from  a finite  set.  This  quantity  characterizes  the 
largest  compression  ratio  that  can  be  achieved  in  accurate  transmis- 
sion of  the  sequence  by  any  finite-state,  fixed  rate  encoder  (and 
decoder).  Coding  theorems  and  their  converses  are  derived  for  an 
individual  sequence  without  any  probabilistic  characterization,  and 
universal  data  compression  algorithms  which  are  asymptotically  opti- 
mal for  all  sequences  over  a given  alphabet  are  introduced. 

The  finite-state  complexity  of  a sequence  plays  a role  similar  to 
that  of  entropy  in  classical  information  theory  (which  deals  with 
probabilistic  ensembles  of  sequences  rather  than  an  individual 
sequence).  For  a probabilistic  source,  the  expectation  of  the  fin- 
ite state  complexity  of  its  sequences  is  equal  to  the  source's  en- 
tropy. The  finite  state  complexity  is  of  particular  interest  when 
the  source  statistics  are  unspecified. 

More  recent  results  which  lead  to  a distortion-rate  theory  for  indi- 
vidual sequences  will  also  be  discussed. 


ERGODIC  AND  MIXING  PROPERTIES  OF  A CLASS  OF  COMPOSITE  SOURCES, 

Robert  J.  Fontana  (Department  of  Electrical  Engineering,  Stanford 
University,  Stanford,  CA) . Sources  whose  probability  distributions 
vary  from  letter  to  letter  are  known  as  composite  sources.  Such  a 
source  can  be  modelled  as  a combination  of  a collection  of  random 
processes  and  an  additional  "switch"  random  process  which  chooses  an 
element  from  the  collection  at  each  instant  of  time.  The  ergodicity 
(weak-mixing,  and  strong-mixing)  of  the  composite  source  is  examined, 
and  in  doing  so,  a one-to-one  correspondence  is  noted  between  a 
regular  composite  source  and  an  inflnte  memory  channel.  Further, 
for  regular  composite  sources,  satisfying  certain  asymptotic  inde- 
pendence constraints,  the  mixing  properties  of  the  source  are  deter- 
mined by  the  mixing  properties  of  the  switch.  Finally  to  better  il- 
luminate these  principles,  examples  are  given  of  composite  sources 
satisfying  the  regularity  and  asymptotic  independence  properties. 
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AN  IN-PLACE  SELF-REORDERING  FFT,  James  K.  Beard  (Honeywell  Inc., 

5303  Shllshole  Ave.  NW,  Seattle,  WA  98107).  The  FFT  algorithm 
inherently  involves  a requirement  that  the  data  be  bit-reverse  ad- 
dress reordered,  either  before  or  after  the  transform  operation. 

If  the  algorithm  is  an  in-place  FFT,  a separate  pass  is  required  to 
perform  this  operation.  As  a typical  example,  a 1024-polnt  radix 
2 algorithm  requires  five  FFT  passes  and  one  reordering  pass,  and 
thus  spends  about  17  percent  of  its  time  in  the  reordering  pass. 

The  use  of  a separate  reordering  pass  can  be  eliminated,  so  that  no 
extra  time  is  required  for  the  reordering  operation  for  an  in-place 
FFT,  by  a technique  using  the  properties  of  the  reordering  operation 
to  perform  an  FFT  pass. 


SYNTHESIZING  LARGE  DFT'S  USING  THE  MUTUAL  PRIME  FACTOR  CYCLIC 
ALGORITHM,  Salvatore  D.  Morgera  (Raytheon  Company,  Submarine  Signal 
Division,  Systems  Laboratory,  Portsmouth,  RI  02871).  The  utiliza- 
tion of  a Mutual  Prime  Factor  Cyclic  Algorithm  (MPFCA)  for  efficiently 
performing  large  Discrete  Fourier  Transforms  (DFT)  is  examined. 

The  mutual  prime  factor  portion  of  the  algorithm  is  originally  at- 
tributed to  L.H.  Thomas,  with  generalization  supplied  by  I.J.  Good; 
the  cyclic  portion  of  the  algorithm  has  been  recently  formalized  by 
S.  Winograd . Two  methods  are  presented  for  implementing  large  size 
transforms  in  a range  of  Interest  from  60-5000  points.  Computational 
complexity  of  the  large  transforms  is  estimated  for  both  methods 
and  compared  with  that  of  the  FFT.  The  computational  savings  are 
found  to  be  quite  substantial  - either  approximately  (^)  or  (h.) 
the  number  of  multiplications  required  by  the  FFT,  depending  on  the 
method  employed.  These  results  bring  to  light  the  importance  of  a 
variable  modulo  arithmetic  and  addressing  capability  for  high  speed 
frequency  domain  processing  for  a multitude  of  important  functions, 
e.g.,  modem  design,  spectral  analysis,  FIR  digital  filtering,  beam- 
forming, etc. 


COMPUTATIONAL  EFFICIENCY  OF  NUMBER  THEORETIC  TRANSFORM  IMPLEMENTED 
FINITE  IMPULSE  RESPONSE  FILTERS,  Thomas  A.  Krlz  and  Dale  F.  Bachman 
(IBM  Federal  Systems  Division,  Owego,  NY  13827).  The  computational 
merits  of  using  Number  Theoretic  Transforms  to  spectrally  implement 
finite  Impulse  response  (FIR)  filters  are  discussed.  It  is  shown 
that,  for  all  but  low  tap  filters,  the  spectral  method  yields  a sig- 
nificant reduction  of  the  multiply  load  and  a modest  reduction  of 
the  add  load  in  comparison  to  the  usual  time  domain  methods. 
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MULTIPLICATIVE  CONVOLUTIONS  AND  FOURIER  TRANSFORMS,  S.  Cohn-Sfetcu, 
(Bell-Northern  Research,  Ottawa,  Canada).  Mellln  convolutions  and 
Hankel  transforms  are  analyzed  as  convolutions  and  Fourier  transforms 
on  multiplicative  locally  compact  abelian  groups,  with  a view  towards 
providing  fast  algorithms  for  use  In  signal  processing. 


THE  SOLUTION  OF  DISCRETE  CONVOLUTIONS  WITH  A BOUNDED  ERROR  CONSTRAINT, 
A.  Arcese  (Electrical  Engineering  Department,  Unlversldad  de  Los 
Andes,  Bogota,  Columbia).  In  this  paper  we  present  a fast  new  al- 
gorithm for  solving  discrete  convolutions  with  a bounded  error 
constraint.  Unlike  least  squares,  the  algorithm  does  not  require 
Inverting  or  forming  the  elements  of  a matrix,  but  rather,  operates 
directly  on  the  data.  The  algorithm  Is  demonstrated  for  the  pre- 
diction equation  on  a synthetic  speech  waveform. 


ERROR  CONTROL  IN  ARRAY  PROCESSORS,  Dhlraj  K.  Pradhan  (Department  of 
Computer  Science,  University  of  Regina,  Regina,  Saskatchewan,  Canada). 
Current  LSI  technology  makes  It  feasible  to  have  computational  faci- 
lity, built  by  using  large  array  processors.  It  Is  not  Impractical 
to  envisage  thousands  of  microprocessors  operating  In  an  array. 

This  paper  proposes  a mode  for  error  control  In  array  processor, 
operating  In  slngle-lnstructlon-multlple-data  stream  (SIMD)  mode. 

In  this  mode  of  operation,  each  processing  element  computes  a single 
function  which  Is  the  Identical  function  for  all  processors  operating 
In  the  array. 

The  error-control  technique  proposed  here  Is  an  extension  of  an 
earlier  technique  (for  error-control  In  blt-wlse  logical  operations). 
The  technique  proposed  In  this  paper  uses  in  a novel  way  generalized 
Reed-Muller  Code  (GRMC)  for  designing  fault-tolerant  array  processor. 
This  design  encodes  data  streams  Into  codewords  In  an  1-th  order 
GRMC.  Then,  these  codewords  are  processed  by  an  array  processor 
which  contains  redundant  processing  elements. 

It  Is  shown  In  this  paper  that  the  resulting  output  vector  forms 
a codeword  In  a higher  order  code.  The  output  vector  Is  then  decoded 
for  error-correctlon/detectlon  and  thus  provides  fault-tolerance. 


DUAL-MODE  LOGIC:  A NEW  REDUNDANCE  TECHNIQUE  FOR  THE  DESIGN  OF 
EASILY  TESTED  LOGIC  CIRCUITS,  S.  DasGupta  (IBM  Corporation, 
Flshklll,  New  York)  and  C.R.P.  Hartmann  and  L.D.  Rudolph  (School 
of  Computer  and  Information  Science,  Syracuse  University,  Syra- 
cuse, New  York).  Traditionally,  logic  circuits  have  been  designed 
with  little  regard  for  the  problem  of  fault  testing.  Designers 
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have  been  concerned  with  minimizing  the  complexity  of  the  circuit 
and  this  was  certainly  understandable  In  the  period  prior  to  the 
development  of  LSI.  When  a logic  circuit  is  small,  there  are  a 
number  of  standard  techniques  for  deriving  tests  to  detect  faults. 
As  the  size  of  the  circuit  grows,  however,  the  number  of  tests  re- 
quired may  Increase  very  rapidly.  Moreover,  the  complexity  of 
finding  the  test  set  may  become  prohibitive.  As  logic  component 
costs  decrease  and  the  difficulty  of  testing  increases,  a point  is 
reached  where  logic  designers  should  begin  to  consider  the  possi- 
bility of  introducing  hardware  redundancy  to  simplify  testing. 

In  this  talk,  we  present  one  such  approach  based  on  the  concept  of 
dual-mode  logic.  The  basic  idea  is  to  use  redundancy  (1)  to  reduce 
the  number  of  tests  required,  and  (2)  to  reduce  the  complexity  of 
deriving  the  tests.  From  a testing  standpoint,  the  ultimate  solu- 
tion would  be  a method  of  designing  logic  networks  which  could  be 
tested  using  a function-independent  test  set  of  minimum  possible 
size,  l.e.,  a test  set  that  is  independent  of  the  logic  function 
and  internal  layout  of  the  network.  For  stuck-at  faults  in  com- 
binational networks  this  is  achieved  through  the  use  of  a family  of 
dual-mode  gates  which  perform  the  required  logic  functions  in  one 
mode  and  are  tested  in  the  other  mode.  It  is  shown  that  in  com- 
binational networks  built  with  dual-mode  gates,  all  patterns  of  t 
or  fewer  stuck-at  faults  can  be  detected  with  only  two  function- 
independent  tests  provided  that  the  combinational  network  has  d - 
2t+2  extra  inputs.  In  sequential  networks  built  with  dual-mode 
gates,  all  patterns  of  t or  fewer  stuck-at  faults  can  be  detected 
with  only  six  function- independent  tests  provided  that  the  sequen- 
tial network  has  d ~ 2t-*-4  extra  inputs. 
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GROUP  THEORETIC  CODES  FOR  BINARY  ASYMMETRIC  CHANNELS,  Serban  D. 

0 Constantin  and  T.  R.  N.  Rao  (Department  of  Computer  Science, 

Southern  Methodist  University,  Dallas,  Texas  75275).  With  the  ad- 
vent of  the  large  scale  Integrated  memory  technologies,  unidirec- 
tional failure  properties  have  been  observed  In  the  functional  be- 
havior of  the  new  memory  cell:  a cell  failure  Is  more  likely  to 
be  a '1  -*•  O'  transition  (or  1-error)  than  a '0  -+■  1'  transition  (or 
0-error)  whose  probability  of  occurrence  Is  low.  In  the  Ideal  case, 
the  probability  of  an  0-error  can  be  assumed  0. 

Starting  with  the  asymmetric  distance  d (X,Y)  between  two  binary 
n-tuples  X and  Y,  a metric  previously  defined  by  one  of  the  authors, 
and  using  the  minimum  asymmetric  distance  requirement  for  a single 
asymmetric-error  (In  our  case  1-error)  correcting  code,  we  Intro- 
duce a new  class  of  codes  suitable  for  the  Ideal  binary  asymmetric 
channel  which  we  will  refer  to  as  'group  theoretic  codes'.  The 
single  asymmetric-error  correcting  capability  of  these  codes  Is 
proved  and  their  superiority  In  Information  rate  over  previously 
developed  single  error  correcting  codes  for  the  Ideal  binary  asym- 
metric channel  or  the  binary  symmetric  channel  Is  established. 

Given  an  Abelian  group  G(+)  with  elements  aQ,aj,...,a^  (a^  the 
Identity  or  zero  element)  we  define  the  sets  VQ,Vp...,V^  to  be 
the  sets  of  binary  n-tuples  whose  scalar  product  with  the  vector 
(apa2,...,a  ) Is  equal  to  aQ,a^ , . . . ,a^,  respectively.  Every  set 
Is  shown  ?o  be  a single  1-error  correcting  code  and  It  will  be 
called  a group  theoretic  code.  For  any  Abelian  group  G(+),  the 
code  constituted  by  the  vectors  of  Vq  Is  proved  to  have  the  highest 
Information  rate  of  all  codes  V^,  1 = l,2,...,n. 

Exact  formulas  are  derived  for  the  size  of  the  sets  generated  by 
the  additive  group  of  any  Galois  field.  A selection  criterion  for 
choosing  the  best  (with  the  highest  Information  rate)  group  theore- 
tic code  of  length  n,  l.e.,  determining  what  Abelian  group  G(+)  of 
order  n+1  generates  the  highest  Information  rate  code  Vq,  Is  pre- 
sented. Many  properties  of  the  group  theoretic  codes  are  proved 
and  several  open  questions  of  Interest  In  the  area  of  group  theore- 
tic codes  are  posed. 


SOME  PROBLEMS  ON  PERMUTATION  GROUP  CODES,  Ge'rard  Cohen  (ENST,  46 
rue  Barrault  75013  Paris,  France)  and  Michel  Deza  (CNRS,  Paris,  France). 
We  consider  mainly  three  directions.  First,  majority  decoding  of 
permutation  codes.  An  Idea  of  Blake,  working  for  sharply  2-transltlve 
groups.  Is  generalized  to  some  other  groups.  Including  Frobenlus 
and  sharply  3-transltlve  ones. 

Secondly,  we  use  permutation  codes  for  source  coding,  computing  dis- 
tortion functions  with  the  Hamming  distance  between  permutations  as 
distortion  criterion,  l.e.,  the  number  of  elements  whose  rank  after 
reproduction  differs  from  their  original  rank.  This  problem  Involves 
weight  enumeration.  Finally,  we  list  some  known  and  now  results  on 
extremal  problems  linked  with  the  minimum  Hamming  distance  of  permu- 
tation codes  versus  such  classical  parameters  of  permutation  groups 
as  degree,  order,  and  degree  of  transitivity. 
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CONFIGURATION  MATRICES  OF  GROUP  CODES  FOR  THE  GAUSSIAN  CHANNEL, 

E.  Biglieri  (Istituto  Elettrotecnlco,  Universlta  dl  Napoli,  Italy 
and  Department  of  System  Science,  University  of  California,  Los 
Angeles)  and  M.  Elia  (Istituto  Matematico,  Politecnico  di  Torino, 
Italy).  Given  an  [M,n]  code  for  the  Gaussian  channel,  i.e.,  a 
collection  of  M vectors  in  Euclidean  n-space,  its  configuration 
matrix  is  the  MxM  matrix  whose  elements  are  the  scalar  products 
between  code  vectors. 

The  aim  of  this  paper  is  to  characterize  the  configuration  matrices 
of  group  codes.  The  following  result  is  proved:  "Let  C be  the  con- 
figuration matrix  of  an  [M,n]  code.  C is  the  configuration  matrix 
of  a group  code  if  and  only  if  a)  the  rows  of  C are  permutations  of 
the  first  row;  and  b)  a matrix  J,  all  of  whose  elements  are  unity 
and  whose  order  j is  not  greater  than  (M-l)I,  exists  such  that  the 
matrix  C(x)j  ((^  the  Kronecker  product)  commutes  with  all  the  matrices 
of  the  right  regular  representation  of  a group  'Q  with  order  | 2?  I “ 
jM. " 

A finite  procedure  is  also  exhibited  that  allows  one  to  test  if  a 
given  matrix  satisfies  requirement  (b) . 


AN  ALGORITHM  FOR  CODING  UNDIRECTED  GRAPHS,  K.  N.  Venkataraman 
(Computer  Group,  Tata  Institute  for  Fundamental  Research,  Bombay, 
India)  and  K.  Thulasiraman  (Computer  Centre,  Indian  Institute  of 
Technology,  Madras,  India).  This  paper  is  concerned  with  the  con- 
struction of  a code  for  undirected  graphs.  The  code,  which  is  a 
string  of  characters,  completely  defines  the  graphs  so  that  two 
graphs  have  the  same  code  if  and  only  if  they  are  isomorphic. 

The  algorithm  to  be  presented  here  is  based  on  the  work  of  Shah, 
Davida,  and  McCarthy.  A counter  example  is  given  to  show  that 
their  algorithm  falls  to  produce  the  optimum  code  for  certain 
graphs.  An  efficient  algorithm  for  the  construction  of  the  opti- 
mum code  is  then  presented.  Though  this  algorithm  was  motivated 
by  the  work  of  Shah  et  al.,  most  of  the  ideas  used  here  are  differ- 
ent from  theirs. 


NEW  CLASSES  OF  SEQUENCES  WITH  GOOD  CORRELATION  PROPERTIES,  Dennis 
A.  Shedd  and  Dilip  V.  Sarwate  (Coordinated  Science  Laboratory  and 
Department  of  Electrical  Engineering,  University  of  Illinois, 
Urbana,  Illinois  61801).  A new  method  is  presented  for  the  design 
of  sequences  with  good  correlation  properties.  Given  a pair  oi  m- 
sequences  of  period  p -1,  we  can  construct  another  sequence  of 
period  p -1  with  the  property  that  the  out-of-phase  autocorrelation 
of  the  new  sequence  is  zero.  These  sequences  are,  in  general, 
multi-valued,  but  proper  choice  of  the  original  m-sequences  en- 
ables us  to  obtain  sequences  that  take  on  values  0,  +1,  and  -1. 

The  new  technique  also  enables  us  to  construct  new  classes  of 
multi-valued  sequences  with  good  cross-correlation  properties. 
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using  either  the  uncorrelated  binary  MacWllllams  sequences  or  the 
Gold  sequences  with  bounded  cross-correlation.  Finally,  we  show 
that  given  two  impulse-equivalent  sequences  of  length  N,  one  can 
design  impulse-equivalent  sequences  of  length  2N-1  without  resort- 
ing to  Huffman's  procedure. 


ON  LINEARIZATION  OF  NONLINEAR  COMBINATIONS  OF  LINEAR  SHIFT  REGISTER 
SEQUENCES,  Tore  Herlestam  (Bankoardsgatan  74,  S-252  60  Helsingborg, 
Sweden).  In  this  paper  a new  and  direct  proof  is  given  of  a 
strengthened  version  of  Hadamard's  multiplication  theorem  in  an 
algebraic  case.  The  analytic  functions  considered  are  rational 
forms  over  a Galois  field,  representing  linear  shift  reg^  .ter 
sequences. 

Let  G(f)  and  G(g)  denote  the  linear  spaces  of  all  shift  register 
sequences  over  the  finite  field  GF(p  ),  with  feedback  polynomials 
f and  g,  respectively.  The  multiplication  theorem  states  that 

1.  there  eslsts  a polynomial  fAg  over  GFCp*"),  depending  on 
f and  g only,  such  that  aAb  - (aicbj^)  e G(fAg)  for  every 
a eG(f),  beG(g) 

2.  every  zero  of  fAg  is  a product  of  a zero  of  f times  a 
zero  of  g, 

3.  if  u is  a zero  of  f and  v of  g,  of  multiplicities  r and 
s,  respectively,  then  uv  is  a zero  of  fAg  of  multiplicity 
i r+s- 1 , 

4.  deg  fAg s deg  f * deg  g, 

5.  if  f and  g have  exclusively  simple  zeroes  and  all  products 
uv  are  different,  then  equality  holds  in  3.  and  4. 

As  applications  of  this  multiplication  theorem,  it  is  easy  to  cor- 
rectly predict  the  maximally  attainable  linear  complexity  of  all 
kinds  of  nonlinear  combinations  of  linear  shift  register  sequences. 


RECENT  RESULTS  ON  ALGEBRAIC  SOFT-DECISION  DECODING,  T.  Y.  Hwang 
(School  of  Computer  and  Information  Science,  Syracuse  University, 
Syracuse,  NY  13210),  N.  Q.  Due  (Telecom  Australia,  Research  Labora- 
tories, 59  Lt.  Collins  Street,  Melbourne,  Victoria,  Australia  3000) 
and  C.  R.  P.  Hartmann  and  L.  D.  Rudolph  (School  of  Computer  and 
Information  Science,  Syracuse  University,  Syracuse,  NY  13210).  In 
this  paper,  we  present  the  most  recent  results  of  a continuing  in- 
vestigation of  algebraic  soft-decision  decoding  of  linear  block 
codes  using  the  dual  code  domain.  It  is  known  that  any  decoding 
algorithm  for  a binary  code  with  minimum  Hamming  distance  d which 
corrects  all  errors  within  an  Euclidean  sphere  of  radius  JaH  (i.e., 
a "maximum-radius"  decoding  algorithm)  is  asymptotically  optimum  for 
the  AWGN  channel.  We  show  that  this  condition  is  not  only  suffi- 
cient for  asymptotic  optimality,  but  is  also  necessary.  It  is  also 
known  that  any  L-step  orthogonalizable  code  can  be  maximum-radius 
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decoded  using  the  orthogonal  parity  checks  and  the  cosine  demodula- 
tion function.  We  extend  this  result  to  Include  a class  of  hard- 
limited  linear  demodulation  functions.  Then  the  relatlonshln  be- 
tween the  Euclidean  radius  of  the  decoding  sphere  and  the  combina- 
torial structure  of  the  parity  check  set  Is  extended  to  nonortho- 
gonal  checks.  Since  algebraic  soft-decision  decoding  using  the 
dual  code  domain  operates  entirely  In  the  domain  of  continuous 
functions  over  the  complex  numbers.  It  lends  Itself  naturally  to 
Iterative  decoding  techniques.  The  hlll-cllmblng  approach  to  de- 
coding Is  described  and  simulation  results  are  given. 
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OPTIMAL  SAMPLING  OF  INDEPENDENT  INCREMENT  PROCESSES,  B.  W.  Stuck 
(Bell  Laboratories)  and  C.  M.  Newman  (Indiana  University).  An  in- 
dependent increment  process  with  one  of  two  sets  of  parameters  is 
observed.  New  necessary  and  sufficient  conditions  are  found  for 
the  probability  measures  of  the  two  processes  not  to  be  mutually 
orthogonal.  A discrete  time  analog  of  this  continuous  time  problem 
is  studied,  where  we  fix  the  number  of  observations  and  study 
Chernoff  type  bounds  on  various  error  probabilities  as  a function 
of  the  sampling  interval.  A partial  solution  is  obtained  to  the 
prob  em  of  choosing  the  optimal  sampling  rate  to  minimize  various 
error  probabilities. 


A PREDICTION  PROBLEM,  David  Slepian  (Bell  Laboratories  and  Univer- 
sity of  Hawaii) . N equally  spaced  samples  of  a bandllmlted  white 
noise  are  combined  linearly  to  form  an  estimate  of  the  next  sample 
that  has  minimum  average  squared  error.  How  does  this  smallest 
error  variance  decrease  with  N and  how  does  it  depend  on  the  in- 
terval between  samples?  An  asymptotic  solution  to  the  problem  is 
obtained  using  properties  of  the  discrete  prolate  spheroidal  wave 
functions . 


PROPERTIES  AND  APPLICATIONS  OF  A USEFUL  CLASS  OF  TWO-DIMENSIONAL 
RANDOM  FIELDS,  R.  W.  Fries  and  J.  W.  Modestino  (Electrical  and  Sys- 
tems Engineering  Department,  Rensselaer  Polytechnic  Institute,  Troy, 
NY  12181).  A useful  class  of  two-dimensional  (2-D)  random  fields 
is  described  and  several  of  their  more  important  properties  dis- 
cussed. This  class  of  random  fields  is  modeled  as  a marked  point 
process  evolving  according  to  a spatial  parameter.  The  autocorre- 
lation and  power  spectral  density  properties  are  investigated  for 
some  special  cases.  Several  applications  are  discussed  including 
the  modeling  of  real-world  Imagery  possessing  Inherent  edge  struc- 
ture. A class  of  efficient  edge  detection  algorithms  is  described 
based  upon  2-D  least  mean-square  Wiener  filtering  concents  where 
the  stochastic  model  for  edge  structure  is  chosen  from  this  class 
of  random  fields.  Additional  applications  drawn  from  the  Imap.e 
coding  area  are  described. 


EXTRAPOLATION  OF  HOMOGENEOUS  RANDOM  FIELDS,  Hlsanao  Ogura  (Depart- 
ment of  Electronics,  Kyoto  Institute  of  Technology,  Matsugasakl, 
Kyoto,  606,  Japan).  An  extrapolation  formula  is  given  for  a homo- 
geneous random  field  with  discrete  parameters,  corresponding  to 
Helson  and  Lowdenslager's  regularity  condition  related  to  the  extra- 
polation In  terms  of  the  "half"  space  values.  The  process  of  extra- 
polation consists  of  two  extrapolation  processes  similar  to  the  one- 
dimensional  prediction.  The  homogeneous  random  field  Is  also  repre- 
sented in  terms  of  the  Innovation  field  associated  with  the  given 
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extrapolation  problem.  The  extrapolation  of  a continuous-parameter 
random  field  can  be  obtained  from  the  discrete-parameter  theory  -just 
as  in  the  prediction  theory  of  a stationary  process.  Several  ex- 
amples of  homogeneous  random  fields  are  studied  using  the  extrapola- 
tion formula  in  order  to  examine  some  specific  aspects,  such  as  the 
Markovian  property. 


ENVELOPE  CONSTRAINED  FILTERS  WITH  UNCERTAIN  INPUT,  Robin  J.  Evans 
and  Antonio  Canton!  (Department  of  Electrical  Engineering,  Univer- 
sity of  Newcastle,  Australia,  2308).  In  the  envelope-constrained 
filtering  problem  with  uncertain  input,  the  filter  impulse  response 
is  optimized  subject  to  the  constraint  that  its  output  lies  within 
a prescribed  envelope  when  the  input  to  the  filter  is  a signal  be- 
longing to  the  set  defined  by  an  input  envelope  constraint.  The 
technique  has  application  in  a variety  of  areas  such  as  robust  radar 
sidelobe  suppression  filter  design,  fixed  equalizer  design  for  com- 
munication channels  and  system  modelling  with  unknown  but  bounded 
noise  on  the  input  and  output  measurements.  The  results  presented 
in  the  paper  constitute  a significant  extension  of  the  recently 
developed  envelope-constrained  filtering  technique  where  the  filter 
impulse  response  is  optimized  subject  to  constraints  on  the  output 
pulse  shape. 


MOMENT  FORMULAS  FOR  THE  NUMBER  OF  SLOPE-CONSTRAINED  LEVEL  CROSSINGS 
BY  A WIDE  CLASS  OF  STOCHASTIC  PROCESSES,  Douglas  R.  Anderson  and 
Daniel  D.  Carpenter  (TRW  DSSG,  Redondo  Beach,  i,A  90278).  Level 
crossings  constrained  by  upper  and  lower  bounds  on  the  slope  at  the 
crossings  arise  in  such  communications  problems  as  transmission 
over  the  channels  with  multipath  fading  or  random  polarization. 

For  fading  channels,  counting  only  the  downcrosslngs  of  a level  £ 
with  slope  exceeding  a given  value  "a"  produces  the  number  of  deep 
fades  characterized  by  £ and  a.  Evaluation  of  the  k-th  far'-'^rial 
moments  of  the  number  of  level  crossings  permits  the  develo,  ...^nt 
of  a series  formula  for  the  distribution  of  the  time  between  the 
n-th  slope-constraint  passage  time  through  readily  adaptable  formulas 
for  unconstrained  level  crossings.  The  authors  have  shown  that  the 
number  of  slope-constrained  upcrossings  (downcrosslngs)  is  a well- 
defined  random  variable  with  first  and  higher  order  moment  formulas 
generalizing  the  unconstrained  results  of  Rice  and  Cramer  and  Lead- 
better  with  the  following  assumptions.  A separable  process  is  de- 
fined on  a finite  time  Interval  with  1)  a derivative  that  exists 
as  a limit  in  probability  and  is  continuous  in  probability  and 
2)  almost  surely  absolutely  continuous  sample  functions.  These 
assumptions  admit  even  separable  processes  with  derivatives  that 
exlst_^and  are  continuous  in  the  mean  of  any  order  as  low  as  one. 

Let  N^  be  the  number  of  these  upcrossings  of  the  level  £ by  X(t) 
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such  that  X'(t)  has  an  infimum  greater  than  a in  some  neighborhood 
of  each  upcrossing.  If  the  Joint  density  function  of  X(t)  and  X'(t) 
exists  and  is  continuous,  then  we  have  for  the  k-th  factorial  moment 


e[n'‘'(n'^- 

a a 


1)...(N 


-^-k+D]  - Jq--*  /q  ^i-'-VxCt),x'(t) 


.yk)dyi...dyk 


• dt. 


where  X(t)  = (X(t  . ,X(tj^)  ) and 
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TREE  ENCODING  OF  SPEECH  AT  8000  BPS,  Stephen  G.  Wilson  (Department 
of  Electrical  Engineering,  School  of  Engineering  and  Applied  Science, 
University  of  Virginia,  Charlottesville,  VA  22901).  Tree  encoding 
of  speech  and  speech-like  stationary  Gaussian  sources  is  examined 
with  an  encoding  rate  of  8000  bps.  Souce  coding  relative  to  the 
selected  distortion  measure  is  accomplished  with  the  M-algorithm, 
which  extends  M survivor  tree  paths  at  each  level  of  the  tree. 

The  paper  focuses  on  two  extensions  of  previous  work  reported  by 
Anderson  et  al.  to  improve  performance  at  the  lesser  bit  rate.  A 
frequency-weighted  error  criterion  is  applied  as  an  attempt  at  im- 
proving the  subjective  quality  of  speech.  Specifically,  a discrete- 
time weighting  network  approximating  the  response  of  the  C-message 
weighting  network  is  developed  and  applied  to  the  coding  of  a 
stochastic  process  having  long-term  speech  statistics.  Tree  search- 
ing with  the  M-algorlthm  using  weighted  error  is  shown  to  produce 
error  spectra  closely  matching  the  theoretically  optimal  allocation, 
and  the  total  weighted  error  is  about  1.5  dB  from  the  rate-distortion 
bound  at  M = 16.  Subjective  evaluation  of  speech  coded  in  this 
fashion  reveals  a preference  for  the  weighted  search  relative  to  the 
standard  squared-error  case. 

Second,  adaptation  of  the  tree  encoder  on  a short-time  Interval  basis 
is  considered  as  a means  of  tailoring  the  source  code  to  the  various 
quasi-stationary  modes  of  speech.  In  conjunction  with  tree  coding 
rs  above,  a slgnal-to-distortlon  power  ratio  of  approximately  15  dB 
was  obtained  for  a typical  sentence.  A sample  of  processed  speech 
at  8000  bps  will  be  played. 


TREE  ENCODING  OF  SPEECH  BASED  ON  SHORT  TERM  AUTOCORREIJVTIONS , 

S.  Mattsson  (Department  of  Electrical  Engineering,  Llnkoplng  Uni- 
versity, Linkoplng,  Sweden).  Low  rate  source  encoders  for  speech- 
like signals  usually  employ  tree  encoding  with  a feedback-structure, 
where  a predictor  in  the  feedback  loop  (the  code  generator)  produces 
an  estimate  of  the  current  input  sample  on  the  basis  of  a digital  input 
from  a search  algorithm  working  on  the  prediction  error. 

All  practical  search  algorithms  (Including  the  Vlterbl  algorithm  and 
the  (M,L) -algorithm)  make  decisions  on  a small  number  of  samples 
(5-25).  Based  on  the  quasistationary  property  of  speech,  we  inves- 
tigate by  simulation  the  performance  of  a binary  (one  bit  per  sam- 
ple) tree  encoding  system,  adapting  to  the  different  modes  of 
statlonarlty . The  code  generator  is  chosen  to  have  an  autoregressive 
structure  where  the  coefficient  vector  belongs  to  a small  set  of 
vectors,  calculated  on  basis  of  the  short-term  autocorrelation  (of 
a typical  speaker).  The  search  algorithm,  consisting  of  the  (M,L)- 
algorlthm,  chooses  for  each  sequence  of  L speech  samples  that  code 
generator  (one  of  2 possible)  which  produces  the  best  (minimum  MSE) 
estimate  of  the  actual  sequence.  Information  (n  bits)  about  the  chosen 
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code  is  transmitted  to  the  decoder  in  each  block  of  L binary  path 
map  digits.  (The  set  of  possible  coefficient  vectors  is  assumed 
to  be  stored  in  the  decoder.)  Simulation  results  for  different 
choices  of  code  generator  ensembles  will  be  presented. 


STACK  ALGORITHM  SPEECH  ENCODING,  S.  Mohan  and  J.  B.  Anderson  (De- 
partment of  Electrical  Engineering  and  Communications  Research 
Laboratory,  McMaster  University,  Hamilton,  Ontario,  Canada,  L8S  4L7). 
In  recent  years,  there  has  been  Increasing  interest  in  the  design 
of  efficient  source  encoding  algorithms.  Source  coding  involves  a 
twofold  design  problem,  design  of  a good  code,  and  design  of  an 
effective  search  algorithm  to  explore  among  the  code  words.  Recent 
experimental  research  has  provided  good  speech  codes;  good  codes  for 
theoretical  sources  are  well  known.  A number  of  code  search  al- 
gorithms have  been  proposed,  and  we  consider  here  the  performance 
of  one,  the  stack  algorithm,  on  speech  waveforms.  In  its  simple 
form,  the  stack  algorithm  is  not  easily  instrumentable , since  its 
stack  can  grow  very  large.  We  thus  Impose  constraints  on  the  width 
and  size  of  the  stack.  Furthermore,  we  divide  the  stack  into  two 
stacks,  a main  stack  and  an  extension  stack,  which  is  periodically 
merged  into  the  main  stack.  This  allows  the  use  of  a computationally 
more  efficient  sort/merge  technique.  Tests  on  actual  speech  with  a 
software- implemented  algorithm  show  a complicated  dependence  on  the 
bias  term  in  the  path  metric,  and  show  that  for  any  fixed  stack  size, 
biases  in  a certain  range  must  be  used.  For  a fixed  distortion  per- 
formance, there  exist  unique  best  stack  size  and  bias  combinations, 
which  optimize  either  storage,  overall  computation  time,  or  nodes 
vlslted/output  released.  Unfortunately,  these  combinations  are  all 
different.  This  behavior  matches  theoretical  results  reported 
earlier  for  the  stack-encoded  binary  1.1. d.  source  with  Hamming 
distortion  measure;  we  conjecture  it  extends  to  many  other  sources. 


SOFT  DECISION  DEMODULATION  FOR  PCM  ENCODED  SPEECH  SIGNALS,  Carl-Erik 
Sundberg  (Telecommunication  Theory,  University  of  Lund,  Pack,  S-220 
07  Lund,  Sweden).  The  effect  of  digital  errors  in  PCM  encoded  speech 
signals  transmitted  over  a noisy  channel  is  reduced  by  using  soft 
decision  demodulation  at  the  receiver.  The  reliability  numbers  sup- 
plied by  the  soft  demodulator  are  used  to  point  out  likely  trans- 
mission errors,  especially  in  the  most  significant  PCM  bits,  \7hen 
a likely  transmission  error  is  identified,  the  corresponding  PCM 
word  is  rejected  by  the  receiver  and  replaced  by  a predictor  esti- 
mate or  an  interpolation  estimate  If  delayed  decisions  are  used. 

We  have  analyzed  soft  decision  demodulation  for  standard  PCM  encoded 
speech  signals  transmitted  over  the  Gaussian  channel  with  coherent 
PSK  (phase  shift  keying)  or  DPSK  (differentially  encoded  PSK  with 
coherent  detection).  A slgnal-to-nolse  ratio  gain  in  Eb^^o 
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order  of  1-2  dB  is  obtained  at  low  Input  signal  levels.  The  gain 
depends  on  the  performance  of  the  predictor  or  alternatively  the 
Interpolator.  No  modifications  of  the  transmitter  are  required  to 
obtain  this  Improvement.  The  comparisons  are  made  with  hard  de- 
cision demodulation. 


MATHEMATICAL  TREATMENT  OF  SPEECH  SIGNALS,  D.  Wolf  and  H.  Brehm 
(Instltut  fur  Angewandte  Physlk  der  Unlversitat  Frankfurt  am  Main, 
Frankfurt  am  Main,  FR  Germany) . Standard  telephone  speech  signals 
may  be  considered  to  be  realizations  of  a stationary  and  ergodlc 
random  process.  The  first  order  amplitude  probability  density  (pd) 
is  well  fitted  by  the  zero  order  modified  Bessel  function.  Thus, 
the  process  can  be  represented  by  the  product  of  two  mutually  in- 
dependent gaussian  random  processes.  Principally,  this  model  allows 
us  to  derive  higher  order  pd's.  The  mathematical  treatment  is  con- 
siderably simplified  by  the  assumption  of  elliptical  invariance. 

This  assumption  has  been  suggested  by  experimental  results  on  the 
statistical  dependencies  between  amplitudes  separated  not  more  than 
5 ms. 

Starting  with  p^  = ^ K^(|z|)  one  obtains  the  higher  order  pd's 

-1/2  -1 

^2  “ ^2  (2^X2)  exp(-r2), 

-1/2  2 -1 

P4  * ' (4^‘r^)  (l+r^)exp(-r^)  etc., 

T 1/2  T 

1 ••••  M » (<q  M^  - det  M. 

These  results  offer  a key  to  an  analytical  treatment  and  the  solu- 
tion of  many  practical  problems  in  the  field  of  digital  processing 
of  speech.  For  example  the  effect  of  linear  prediction  and  gain 
control  on  the  first  order  pd  will  be  discussed  in  more  detail. 


RECOGNITION  OF  WORDS  FROM  A REGULAR  LANGUAGE  IN  THE  PRESENCE  OF 

NOISE,  R.  L.  Kashyap  (School  of  Electrical  Engineering,  Purdue 

University,  West  Lafayette,  IN  A7907) . Consider  an  alphabet  A - 

{a^,  ...,  a } made  of  m distinct  symbols  and  a set  B » {x^,  ..., 

X } where  each  word  x.  is  a concatenation  of  symbols  from  A with 
m^  1 
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no  adjacent  symbols  Identical.  Suppose  we  are  given  a string  z,  a 
concatenation  of  symbols  in  A in  which  adjacent  symbols  could  be  the 
same,  z is  a noise  corrupted  version  of  some  xt  B,  where  the  noise 
can  delete  some  symbols  of  x or  Insert  some  symbols  into  x or  re- 
place some  symbols  of  x by  other  symbols.  Our  problem  is  to  find 
the  word  in  B which  is  "nearest"  to  the  string  z according  to  a 
suitable  distance  measure  D(z,x).  The  available  distance  measures 
between  strings  are  not  appropriate  for  this  problem  in  view  of  the 
special  condition  on  the  members  of  B.  We  define  a new  distance 
measure  between  z and  x in  terms  of  the  2 operations  of  deletion 
and  replacement  (no  Insertion)  operations  needed  to  convert  z into 
X,  the  weights  attached  to  the  operation  depending  on  the  symbol 
involved.  The  weights  are  chosen  so  that  D(z,x)  can  be  interpreted 
as  the  negative  of  the  likelihood  function  that  z could  have  been 
generated  by  x. 

This  distance  measure  is  particularly  appropriate  in  some  speech 
recognition  problems.  We  given  an  algorithm  to  determine  the  x 
which  minimizes  the  value  D(z,x).  Unlike  some  of  the  algorithms 
in  the  literature,  we  need  to  compute  D(z,x)  only  for  a small  number 
of  X belonging  to  B. 


ESTIMATION  OF  PHONEMIC  CONFUSION  USING  THE  CONCEPT  OF  MAHALANOBIS 
DISTANCE,  S.  Tazakl,  H.  i^sawa,  Y.  Yamada  and  T.  Gotoda  (Ehime  Uni- 
versity, Matsuyama,  790,  Japan).  The  problem  of  estimating  quanti- 
tatively the  possibility  of  phonemic  confusion  is  described.  This 
problem  originates  from  the  fact,  which  has  already  been  reported 
in  various  journals,  that  the  fluctuation  of  cues  which  results  from 
differences  in  following  vowels  and  differences  in  speakers  affects 
listeners'  ability  to  recognize  and  distinguish  among  phonemes. 
Further  we  have  discovered  that  we  need  to  make  use  of  several  cues 
in  the  frequency  domain  as  well  as  in  the  time  domain  for  doing 
reliable  discrimination  of  phonemes.  In  the  first  place,  the  con- 
cept of  Mahalanobls  distance,  one  of  the  Multivariate  Analysis 
methods,  is  applied  to  set  up  an  efficient  metric  between  two  dif- 
ferent Isolated  syllables  because  this  distance  is  easily  calculated 
by  using  a 'calculus  of  variation'  so  as  to  minimize  the  variance 
within  each  syllable  and  maximize  the  variance  between  the  two  syl- 
lables. Next,  the  syllables  which  belong  to  the  same  phoneme  are 
arranged  systematically. 

We  regard  the  deviation  from  the  average  metric  value  to  be  the 
range  of  fluctuation  in  a phoneme.  We  also  regard  the  average  dis- 
tance between  the  different  phonemes  to  be  the  range  of  Interference 
with  each  other  phoneme.  As  a result,  in  the  case  of  Japanese  stop 
consonants,  it  was  found  that  over  80  percent  of  our  estimates  could 
be  shown  to  correspond  with  actual  data,  and  in  addition,  for  the 
first  time  to  our  knowledge  the  direction  of  confusion  could  be 
estimated,  at  the  rate  of  about  80  percent. 
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SHANNON’S  "PROOF"  OF  THE  NOISY  CODING  THEOREM,  James  L.  Massey 
(Department  of  Electrical  Engineering  and  Computer  Science, 
Massachusetts  Institute  of  Technology,  Cambridge,  MA  02139). 

It  Is  often  said,  particularly  by  mathematicians,  that  Shannon 
nevery  really  "proved"  his  Noisy  Coding  Theorem  which  asserted  that 
the  decoding  error  probability  can  be  made  arbitrarily  small  at  any 
code  rate  R less  than  channel  capacity  C.  It  will  be  shown  on  the 
contrary  that  Shannon's  original  "proof,"  In  the  case  of  a discrete 
memoryless  source  (DMS)  and  a discrete  memoryless  channel  (DMC) , 
can  be  made  completely  rigorous  using  only  methods  well  within  the 
grasp  of  an  average  undergraduate  engineering  student.  It  will  be 
argued  that  the  dissatisfaction  with  Shannon's  "proof"  Is  more 
likely  traceable  to  the  failure  to  see  that  Shannon's  proof  con- 
siders two  quite  distinct  situations  [namely,  (1)  a "gedanken"  ex- 
periment to  determine  the  good  candidates  for  code  words,  and  (2)  a 
coding  scheme  for  transmitting  the  DMS  through  the  DMC],  than  to 
the  "Incompleteness"  of  Shannon's  "proof." 


A NEW  LOOK  AT  EXPONENTIAL  ERROR  BOUNDS  FOR  MEMORYLESS  CHANNELS, 

I.  Cslszar,  J.  Korner,  and  K.  Marton  (Mathematical  Institute  of  the 
Hungarian  Academy  of  Sciences,  H-1053  Budapest,  Reiltanoda  u. 13-15, 
Hungary).  We  prove  a nop-probablllstlc  lemma  on  the  existence  of 
a "balanced"  subset  of  X of  size  exp(«R).  With  this  codeword  set, 
the  mutual  Information  distance  decoding  rule  achieves  Gallager's 
and  Forney's  exponential  error  bounds  universally  for  DMC's  with 
Input  alphabet  X when  restricting  attention  to  fixed  composition 
codes.  The  main  Interest  of  these  results  Is  that  both  the  codeword 
set  and  the  decoding  rule  are  Independent  of  the  channel.  This 
means.  In  particular,  that  for  the  class  of  DMC's  with  critical  rate 
for  the  given  composition  less  than  R,  there  exist  asymptotically 
optimal  universal  codes  of  rate  R. 

Further  applications  of  the  lemma  as  well  as  new  bounds  on  the  pro- 
bability of  decoding  error  on  a DMC  with  noiseless  feedback  will  be 
discussed. 


RELIABILITY  FUNCTION  OF  A DMC  AT  RATES  ABOVE  CAPACITY,  J.  Korner 
(Mathematical  Institute,  Hungarian  Academy  of  Sciences).  It  Is 
known  as  Wolfowltz's  strong  converse  to  the  coding  theorem  for  a 
discrete  memoryless  channel  (DMC)  that  at  rates  above  capacity  the 
probability  c(R,n)  of  correct  decoding  of  the  "best"  n-length  block 
code  of  rate  at  least  R tends  to  zero.  Here, 

c(R,n)  A max  min  c 

e 5ee  X 
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where  the  maximum  is  taken  over  all  codes  e of  rate  at  least  R and 
block-length  n,  and  c Is  the  probability  that  no  error  occurs  when 
the  codeword  x of  the-code  C is  transmitted.  Arimoto  has  given  an 
exponential  upper  bound  on  c(R,n)  having  a form  similar  to  Gallager' 
expression  of  the  random-coding  exponent. 

In  this  paper  we  give  a simpler  derivation  of  the  ssune  bound  in  the 
now  familiar  divergence  extremum  form  and  show  that  the  bound  is 
tight  at  all  rates  above  capacity.  The  result  is  that 

11m  - - log  c(R,n)  = min  min  2 P(x)D(W(' |x) 1 1 W(*|  x)) 

A „ 

n-+«>  p w x€X 

+ [R-KP.W)]"^ 

where  the  first  minimum  refers  to  all  probability  distributions  P 
on  the  input  alphabet  X of  the  DMC^W:X-*-Y,  and  the  second  minimum  is 
for  all  "test  channels"  W:X-*-Y.  D(q1  |q)  is  the  informational  diver- 
gence (or  discrimination)  and  [tJ'^Amax  [t,0]. 

This  result  has  theoretical  interest  inasmuch  as  it  shows  that  for 
R>C  there  is  no  critical  rate,  i.e.,  no  "break  in  the  behavior  of 
the  average  code"  occurs. 


BLOCK  CODING  FOR  DISCRETE  STATIONARY  d-CONTINUOUS  NOISY  CHANNELS, 
Robert  M.  Gray  and  Donald  S.  Ornstein  (Electrical  Engineering  De- 
partment and  Mathematics  Department,  Stanford  University,  Stanford, 
CA  94305).  A new  class  of  discrete  stationary  noisy  channels  with 
memory  termed  d-continuous  channels  is  introduced  and  shown  to  in- 
clude all  stationary  discrete  channels  for  which  coding  theorems 
exist.  Roughly  speaking,  a d-continuous  channel  is  such  that  the 
effect  of  the  "past"  and  "future"  inputs  on  the  "present"  outputs 
dies  out  asymptotically  as  measured  in  a d or  average  Hamming  dis- 
tance sense.  This  is  weaker  than  the  corresponding  notions  of 
Pfaffelhuber,  Kadota,  and  Wyner,  who  require  that  probabilities  of 
all  n-tuples  be  close;  that  is,  closeness  in  a variational  or  dis- 
tribution sense. 

General  block  channel  coding  and  block  joint  source  and  channel 
coding  theorems  are  proved  for  stationary  d-continuous  channels, 
and  various  definitions  of  channel  capacity  are  compared. 
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10  RANDOMNESS  IN  DISCRETE  CHANNELS  WITH  MEMORY,  P.  C.  Shields  (Univer- 

sity of  Toledo,  Toledo,  OH  43606)  and  D.  L.  Neuhoff  (University  of 
Michigan,  Ann  Arbor,  MI  48109).  A result  on  the  nature  of  the 
randomness  in  discrete  channels  with  memory  is  presented.  It  Is 
shown  that  certain  channels,  including  finite-memory  channels,  may 
be  decomposed  into  a memoryless  channel  followed  by  a filter. 
Equivalently,  it  is  shown  that  such  channels  may  be  simulated  with 
an  external,  memoryless  source  of  randomness  and  a sliding-block 
code.  The  channels  considered  here  are  d-contlnuous  (in  the  sense 
of  Gray  and  Ornstein)  and  conditionally  almost  block  Independent 
(a  notion  related  to  very  weak  Bernoulli  random  processes) . Chan- 
nels in  this  class  have  almost  finite  input  memory  and  almost  finite 
output  memory.  A measure  of  channel  similarity  called  the  d-channel 
distance  is  Introduced  and  it  is  shown  that  the  above  channels  are 
precisely  the  class  of  channels  which  can  be  well  approximated  by 
finite-memory  channels.  We  conclude  that  the  randomness  in  such 
channels  is  in  no  way  different  than  the  randomness  in  a memory less 
channel  or  memoryless  source  and  channels  not  of  this  type  cannot 
be  well-modelled  by  finite-memory  channels. 


METRICS  MATCHED  TO  THE  DISCRETE  MEMORYLESS  CHANNEL,  Gerald  Seguin 
(Department  of  Electrical  Engineering,  Royal  Military  College, 
Kingston,  Ontario,  Canada).  A metric  D is  said  to  be  matched  to  a 
discrete  memoryless  channel  (DMC)  if  it  is  the  sum  on  its  coordi- 
nates of  a single  letter  metric  and  if  the  maximum  likelihood  de- 
coder is  a minimum  D-distance  decoder.  Necessary  and  sufficient 
conditions  on  the  transition  probabilities  of  a DMC  for  the  exis- 
tence of  a matched  metric  are  derived.  In  the  case  of  the  binary 
DMC  these  are  equivalent  to  the  channel  being  symmetric.  Explicit 
transition  probabilities  are  given  for  a large  class  of  ternary  DMC 
with  an  associated  matched  metric. 


THE  GEOMETRY  OF  PROBABILITY  SPACE,  Richard  E.  Blahut  (International 
Business  Machines,  Inc.,  Owego,  NY  13827).  The  space  of  all  prob- 
ability distributions  on  a finite  set  of  size  K elements  is  made 
into  a vector  space  by  suitable  definitions  of  vector  addition  and 
scalar  multiplication.  Distance  and  size  measures  are  studied 
within  the  setting  of  this  vector  space.  This  structure  yields 
bounds  on  the  performance  of  testers  of  binary  or  multiple  hypo- 
theses and  of  channel  codes. 
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BOUNDS  ON  UPDATE  ALGORITHMS,  Richard  A.  Flower  (Coordinated  Science 
Laboratory,  University  of  Illinois,  Urbana,  IL  61801).  Updates  In 
a computer- implemented  data  system  are  Investigated  using  a data 
model  developed  by  Ellas  and  Welch.  In  the  model  a data  base  Is 
recorded  by  storing  a representative  binary  string  In  computer 
memory.  A retrieval  question  Is  answered  by  reading  individually 
addressable  memory  cells,  each  of  which  stores  a single  binary  value 
of  the  representative  string.  An  update  to  a new  data  base  Is  made 
by  an  algorithm  which  reads  and/or  writes  at  some  memory  cells  and 
halts  when  the  string  In  memory  is  one  of  the  strings  representing 
the  new  data  base. 

Memory  costs  and  retrieval  costs  are  two  characterizations  of  sys- 
tem performance.  These  costs  have  been  investigated  by  Elias.  Here 
we  investigate  update  costs,  which  are  another  important  character- 
ization of  system  performance. 


STOCHASTIC  ERROR  ANALYSIS  OF  SPLINE  APPROXIMATION,  H.  L.  Welnert 
(Electrical  Engineering  Department,  Johns  Hopkins  University, 
Baltimore,  MD  21218),  G.  S.  Sldhu  (Instltuto  de  Ingenleria,  Uni- 
versity Nacional  de  Mexico,  Mexico  20,  D.F.)  and  R.  H.  Byrd  (Math. 
Sciences  Department,  Johns  Hopkins  University,  Baltimore,  MD  21218). 
In  order  to  use  traditional  spline  approximation  error  bounds,  one 
needs  at  the  very  least  a tight  upper  bound  on  some  nonlinear  func- 
tional of  the  unknown  function  producing  the  data.  In  most  practi- 
cal problems,  however,  this  information  is  not  available,  and  thus 
these  bounds  cannot  be  computed.  The  most  one  can  do  in  this  situ- 
ation is  bound  a normalized  error.  This  computable  upper  bound  is 
In  fact  the  mean-square  error  of  an  associated  least-squares  esti- 
mation problem  whose  statistics  are  determines  by  the  type  of  spline 
used.  The  bound  Is  Independent  of  the  data  and  can  thus  be  used  to 
develop  optimal  sampling  schemes. 


FITTING  CURVES  TO  DETERMINISTIC  DATA,  H.  L.  Welnert  and  R.  H.  Byrd 
(Johns  Hopkins  University,  Baltimore,  MD  21218)  and  G.  S.  Sldhu 
(Instltuto  de  Ingenleria,  University  Nacional  de  Mexico,  Mexico  20, 
D.F.).  Many  of  the  optimal  curve  fitting  and  approximation  problems 
arising  in  numerical  analysis  have  the  same  structure  as  certain  es- 
timation problems  Involving  random  processes.  This  structural  cor- 
respondence, whose  development  depends  on  results  from  the  theories 
of  reproducing  kernels,  linear  systems,  and  stochastic  realization, 
has  many  useful  consequences.  For  one  thing,  it  tells  us  that  op- 
timal curve  fits  (splines)  are  sample  functions  of  linear  least- 
squares  estimates.  As  a result,  recursive  estimation  techniques 
can  be  used  to  solve  curve  fitting  problems  and  to  provide  tight 
upper  bounds  for  fit  error.  In  addition,  the  correspondence  yields 
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Interesting  Information  on  the  sample  function  properties  of  least- 
squares  estimates  of  certain  random  processes.  Previous  work  has 
dealt  with  splines  determined  by  differential  operators;  these 
splines  are  sample  functions  of  estimates  of  autoregressive  pro- 
cesses (generated  by  all-pole  systems) . The  present  work  examines 
splines  determined  by  certain  integro-dlf f erential  operators;  these 
splines  are  sample  functions  of  autoregressive-moving  average  pro- 
cesses (generated  by  systems  with  zeros). 


SQUARE-ROOT  ALGORITHMS  FOR  PARALLEL  PROCESSING  IN  OPTIMAL  ESTIMATION, 
M.  Morf,  J.  Dobbins,  B.  Frledlander  and  T.  Kallath  (Information  Sys- 
tems Laboratory,  Stanford  University,  Stanford,  CA  94305).  We  give 
explicit  algorithms  in  square-root  form  which  allow  measurements  for 
the  standard  state  estimation  problem  to  be  processed  in  a highly 
parallel  fashion  with  little  communication  bet^'een  processors.  Af- 
ter this  preliminary  processing,  blocks  of  measurements  may  be  in- 
corporated into  state  estimates  with  essentially  the  same  computa- 
tion that  usually  accompanies  the  incorporation  of  a single  measure- 
ment. Other  results  of  this  formulation  of  the  estimation  problem 
Include  square  root  doubling  formulas  for  calculating  steady  state 
covariance  with  constant  models  and  an  extension  of  the  class  of 
problems  for  which  Chandrasekhar-type  algorithms  offer  computation 
reductions  to  include  piecewise  constant  systems  with  arbitrary 
initial  conditions. 


LADDER  FORMS  FOR  ESTIMATION  AND  DETECTION,  M.  Morf,  D,  T.  Lee  and 
A.  Vieira  (Information  Systems  Laboratory,  Stanford  University, 
Stanford,  CA  94305).  Many  efficient  algorithms  in  estimation 
theory  have  been  based  on  certain  canonical  forms.  For  Instance, 
the  observer  canonical  form  is  one  that  has  been  widely  used  in  al- 
gorithms involving  state  estimation  via  Riccati  and  Chandrasekhar- 
type  equations.  One  class  of  canonical  forms  that  provide  the  most 
promising  alternative  are  ladder  (or  lattice)  canonical  forms.  These 
ladder  forms  have  the  lowest  computation  and  storage  requirements 
and  their  nice  property  of  stability  and  minimum-phase  can  be  checked 
by  inspection  of  the  so-called  "reflection  coefficients**.  These  co- 
efficients can  be  identified  with  the  partial  correlation  (PARCOR) 
coefficients  from  the  stochastic  point  of  view,  and  they  also  have 
physical  significance  related  to  the  scattering  theory  of  waves. 

It  was  shown,  for  example  by  Itakura  and  Wakita,  that  the  ladder  co- 
efficients of  an  autoregressive  (AR)  model  can  be  computed  via 
Levinson's  recursions  involving  the  forward  and  backward  predictors 
(a(z)  and  b(z)).  The  ladder  coefficients  are  in  one-to-one  corre- 
spondence with  the  correlation  sequence,  as  well  as  to  the  a(z)  and 
b(z)  polynomials.  Therefore  it  should  be  possible  to  compute  the 
ladder  coefficients  without  the  explicit  use  of  the  predictor  co- 
efficients. The  fact  being  that  the  computation  for  the  ladder 
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coefficients  involves  an  inner  product  of  an  input  data  sequence, 
e.g.  the  correlation  sequence,  with  the  impulse  response  of  the 
predictor,  a(z),  it  becomes  clear  that  the  ladder  coefficients  are 
sufficient  to  parameterize  the  predictor  and  therefore  the  algorithms 
to  determine  the  ladder  coefficient  do  not  necessarily  require  the 
explicit  use  of  a(z). 

We  show  here  how  these  algorithms  can  be  obtained  as  a natural  ex- 
tension of  the  Fast  Cholesky  or  Chandrasekhar  equations  similar  to 
their  derivations  as  an  extension  of  Levinson's  algorithms. 

Then  we  show  how  to  extend  these  algorithms  to  the  moving-average 
(MA)  processes  via  the  use  of  feedback  ladder  forms.  Especially 
for  these  feedback  forms,  the  property  of  stability  by  inspection 
is  very  useful.  We  also  consider  autoregressive-moving-average 
(ARMA)  models  by  combining  these  two  results. 

All  these  algorithms  can  be  generalized  to  multichannel,  nonstation- 
ary and  multi-dimensional  cases.  We  shall  discuss  some  Interesting 
applications  of  these  ladder  forms  to  problems  in  estimation  and 
detection. 


ON  SOME  COMPLEXITY  PROBLEMS  IN  COMMUNICATION  THEORY,  K.  Yao  (De- 
partment of  System  Science,  University  of  California,  Los  Angeles, 
CA  90024).  Advances  in  microelectronics  have  made  the  study  of 
efficient  algorithms  for  digital  processing  a topic  of  Interest. 

A survey  of  known  results  is  made  on  minimal  multiplications  needed 
in  performing  inner  product  of  vectors  and  matrix-vector  multipli- 
cations with  and  without  using  preconditioning  requirements.  New 
results  involving  minimal  multiplication  realizations  are  obtained 
for  several  detection  and  extended  generalized  Wiener  filter  prob- 
lems. In  many  cases,  there  can  be  considerable  reduction  in  the 
number  of  multiplications  with  only  small  decrease  of  performance 
or  increase  of  programming  cost. 


ON  THE  USE  OF  ASSOCIATIVE  MEMORIES  FOR  PATTERN  RECOGNITION  AND  IN- 
FORMATION PROCESSING.  Yeh-Han  Pao  and  W.  L.  Schultz  (Case  Western 
Reserve  University,  Cleveland,  OH  44106).  The  characteristics  of 
associative  or  content  addressable  memories  which  may  be  Implemented 
with  parallel  processing  techniques  and  offer  efficient  memory  space 
utilization  are  discussed  in  the  context  of  pattern  recognition  and 
adaptive  control  applications. 
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MAXIMUM  POSTERIOR  PROBABILITY  DEMODULATION  OF  ANGLE-MODULATED  SIG- 
NALS, D.  W.  Tufts  (Electrical  Engineering  Department,  University  of 
Rhode  Island,  Kingston,  R1  02881)  and  J.  T.  Francis  (Naval  Under- 
water Systems  Center,  Newport,  RI  02840).  Certain  results  of 
functional  analysis  are  used  to  solve  the  problem  of  Maximum  Pos- 
terior Probability  (MPP)  demodulation  of  angle-modulated  signals. 

A sampled  data  version  of  the  problem  is  considered,  in  which  addi- 
tive Gaussian  noise  and  a Gaussian  modulating  signal  are  assumed. 

The  results  of  this  paper  can  be  considered  as  a numerical  method 
for  solving  the  nonlinear  integral  equation  which  specifies  the 
necessary  condition  for  MPP  estimation  for  the  corresponding 
continuous- time  problem. 

The  results  can  be  applied  to  the  generation  of  a partially  coherent 
reference  for  improving  signal  detection. 


PRE-PROCESSING  OF  SIGNALS  FOR  A CLASS  OF  CDM  PARALLEL  DATA  TRANS- 
MISSION SYSTEMS,  F.  Ghanl  (Department  of  Electrical  Engineering, 
Aligarh  University,  Aligarh  202001,  India).  The  paper  presents  a 
technique  for  the  processing  of  signals  at  the  transmitter  for  a 
class  of  CDM  parallel  systems  suitable  for  the  transmission  of 
digital  data  over  time-invariant  or  slowly  time-varying  channels. 
The  processing  at  the  transmitter  is  such  that  it  maximizes  output 
signal-to-noise  ratio  and  no  detection  process  is  needed  at  the 
receiver.  It  is  shown  that  the  performance  of  the  scheme  in  the 
presence  of  white  Gaussian  noise  is  similar  to  that  of  a serial 
arrangement  where  all  the  processing  is  achieved  at  the  receiver. 
The  proposed  technique  combines  the  advantages  of  FDM  parallel  sys- 
tems and  serial  systems  used  in  data  transmission. 


ERROR  PROBABILITY  FOR  ZERO-CROSSING  DETECTION  OF  DIGITAL  FM,  Hiiseyin 
Abut  (Bogazicl  University,  Istanbul,  Turkey).  An  upper  bound  is 
presented  for  evaluating  the  error  probability  of  a binary  FM  system 
subject  to  additive  Gaussian  noise  (AGN).  We  consider  in  this  work 
a binary  FM  system  that  employs  a continuous-phase  FSK  modulator  and 
a zero-crossing  detector.  The  zero-crossing  (ZC)  detector  counts 
the  number  of  zero-crossings  of  the  signal  imbedded  in  noise.  If 
the  count  exceeds  a threshold  N then  a "space"  is  announced;  other- 
wise, the  decision  is  a "mark". 

An  erroneous  detection  results  due  to  the  presence  of  noise  either 
when  an  extra  ZC  occurs  or  the  counter  misses  a ZC.  There  are  four 
equally  probable  cases  leading  to  detection  errors,  one  of  them 
being  that  of  having  no  ZC  when  a mark  is  transmitted  and  the  in- 
stantaneous phase  of  the  sinusoidal  process  indicates  that  a ZC 
should  have  been  present.  Upper  bounds  to  the  error  probability 
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are  derived  for  all  these  cases.  However,  expressions  for  the 
bounds  are  found  to  be  intractable  and  the  resulting  triple  integrals 
have  been  computed  numerically.  The  obtained  upper  bounds  are  re- 
markably good.  In  fact,  if  the  baud  rate  is  1200  bits/sec  and  the  mark 
and  space  frequencies  are  1300  and  2100  Hz,  respectively,  then  the 
upper  bound  on  error  probability  is  2x10“^  for  a slgnal-to-nolse 
ratio  (SNR)  of  12  dB.  This  value  is  lower  by  a factor  of  ten  in 
comparison  with  any  other  results  available  in  the  data  communica- 
tions literature. 


OPTIMUM  PCM  CODES  FOR  DPSK,  Carl-Erik  Sundberg  (Telecommunication 
Theory,  University  of  Lund,  Fack,  S-220  07  Lund,  Sweden).  The  ef- 
fect of  digital  errors  in  PCM  systems  for  speech  signals  depends  on 
the  PCM  code  used.  The  standard  binary  folded  PCM  code  is  superior 
to  the  natural  binary  code  but  it  is  not  optimum.  This  paper  con- 
siders PCM  codes  especially  designed  to  be  insensitive  to  the  error 
patterns  produced  by  differentially  encoded  phase  shift  keying  (DPSK). 
It  is  concluded  that  the  dynamic  range  with  specially  designed  PCM 
codes  is  considerably  extended  compared  to  standard  binary  folded 
PCM.  Special  simplified  Interleaving  schemes  with  optimum  PCM  codes 
for  coherent  PSK  are  also  considered.  The  dynamic  behavior  of  vari- 
ous PCM  codes  is  calculated  and  compared,  e.g.,  asymptotically  opti- 
mum PCM  codes  for  DPSK,  binary  folded  PCM  with  and  without  inter- 
leaving, asymptotically  optimum  PCM  codes  for  PSK  (coherent),  etc. 

We  conclude  that  the  best  scheme  for  DPSK  is  an  optimum  PCM  code 
for  PSK  and  simplified  interleaving.  The  reencodlng  from  standard 
binary  folded  to  the  new  PCM  codes  can,  for  example,  be  implemented 
by  means  of  a read  only  memory. 


COSYDAI  - A NEW  INTEGRATED  COMMUNICATION  SYSTEM  WITH  DATA  COMPRES- 
SION AND  ERROR  CONTROL,  M.  Barduccl,  G.  Benelll,  V.  Cappelllni,  and 
E.  Del  Re  (Instltuto  dl  Elettronica,  Facolta'  dl  Ingegneria,  Universita  di 
Firenze,  Italy).  Communication  systems  using  both  data  compression 
and  error  control  coding  are  considered.  A software  package  to 
simulate  systems  using  the  two  operations  in  cascade  is  described. 

A new  integrated  compression-coding  strategy,  called  COSYDAI 
(compression  with  SYnchronization  control  and  DAta  Interpolation), 
is  proposed,  in  which  a suitable  synchronization  is  added  to  com- 
pressed data  and  special  controls  are  performed  on  the  received 
data  for  error  correction.  By  examining  the  received  synchroniza- 
tion words,  decisions  are  taken  regarding  the  errors  introduced  by 
the  communication  channel  and  the  words  estimated  with  errors  are 
reconstructed  through  interpolation  operations.  Results  obtained 
by  using  the  above  simulation  package,  suitably  modified  with 
prediction  algorithms,  show  how  in  several  channel  noise  conditions 
(as  with  burst  noise)  the  COSYDAI  system  has  higher  efficiency  than 
the  more  standard  systems  which  perform  the  operations  of  data 
compression  and  error  correction  in  cascade. 
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ERROR  RATE  PERFORMANCE  OF  A FADING  MULTICHANNEL  SYSTEM,  Howard  E. 
Nichols  (GTE  Sylvanla,  Electronic  Systems  Group,  77  "A"  Street, 
Needham  Heights,  MA  02194).  This  paper  considers  the  problem  of 
digital  communication  over  a channel  whose  performance  is  limited 
by  fading  and  interchannel  interference.  Using  the  optimum  maxi- 
mum likelihood  detector,  bit  error  rate  performance  is  calculated 
for  flat  Rayleigh  fading  channels  corrupted  by  interchannel  inter- 
ference and  additive  white  Gaussian  noise.  It  is  shown  that  the 
performance  for  an  N-channel  system  is  similar  in  form  to  the  per- 
formance of  an  N-diversity  system  using  maximal-ratio  combining. 


TRELLIS  CODING  WITH  EXPANDED  CHANNEL-SIGNAL  SETS,  G.  Ungerboeck 
(IBM  Research  Laboratory,  CH-8803  Rueschlikon,  Switzerland).  In 
this  paper  further  progress  in  trellis  coding  with  expanded  sets 
of  multilevel/phase  channel  signals  is  reported.  This  coding  ap- 
proach aims  at  improving  error  performance  of  data  links  without 
sacrificing  data  rate  or  requiring  more  bandwidth  or  power,  by  en- 
coding information  at  given  rate  with  a redundant  number  of  channel 
signals.  Assuming  soft  ML  decoding  in  the  receiver,  the  objective 
in  designing  codes  is  maximizing  free  Euclidean  distance  between 
code  sequences  of  multi level /phase  channel  signals.  First  heuristl- 
cally  designed  codes  with  coding  gains  of  3-4  dB  have  been  presented 
at  the  last  Symposium.  Now  the  development  of  more  powerful  codes 
is  reported  which  can  be  viewed  as  binary  convolutional  codes  of 
rate  R“m/ (nH-1) , where  m is  the  number  of  bits  to  be  transmitted  per 
baud,  followed  by  mapping  mfl  coded  bits  into  an  expanded  set  of 
2*”^^  channel  signals.  For  a specific  mapping,  termed  mapping  by 
set  partitioning,  a new  measure  for  distance  between  binary  code 
sequences  was  defined,  which  efficiently  lowerbounds  the  Euclidean 
distance  between  the  corresponding  channel-signal  sequences.  The 
lowest  bound  is  always  achieved  between  some  channel-signal  sequences 
and  hence  equals  the  free  Euclidean  distance  between  these  sequences. 
Based  on  the  new  distance  measure,  a search  algorithm  for  binary 
convolutional  codes  was  developed,  by  which,  in  conjunction  with  the 
mapping  rule,  codes  with  coding  gains  up  to  6 dB  were  found  for  a 
wide  range  of  practical  multilevel/phase  modulation  forms. 
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SOME  STATISTICAL  PROPERTIES  OF  THE  MONOPULSE  RATIO,  Irving  Ranter 
(Raytheon  Company,  Bedford,  MA) . The  monopulse  ratio  is  treated 
under  conditions  of  multiple  non-fluctuating  targets  and  correlated 
Gaussian  Interference  between  the  difference  and  sum  channels. 

Three  theorems  concerning  respectively  the  mean,  the  variance  and 
the  first  absolute  central  moment  are  derived.  The  theorems  are 
valid  for  arbitrary  signal-to-interference  ratio;  their  proofs  do 
not  require  knowledge  of  the  probability  density  function  (p.d.f) 
of  the  monopulse  ratio.  The  results  are  compared  to  the  approxi- 
mate procedures  currently  employed  by  radar  engineers  when  the  sig- 
nal-to-interference ratio  is  large  or  when  the  interference  is 
caused  by  jamming  signals.  When  the  signal-to-lnterf erence  ratio 
vanishes,  the  results  present  the  mean  and  "spread"  of  the  p.d.f. 
of  the  monopulse  ratio  when  the  radar  confronts  multiple  jammers. 

The  appendix  corrects  a Soviet  publication  which  gives  the  p.d.f. 
of  the  monopulse  ratio  as  an  infinite  series  of  hypergeometric 
functions.  It  also  presents  a closed  form  solution  by  the  author 
which  involves  only  Bessel  functions  of  order  zero  and  one.  When 
the  first  absolute  central  moment  is  calculated  using  each  of  these 
representations,  theorem  III  is  verified. 


DETECTION  AND  PARAMETER  ESTIMATION  OF  CLOSELY  SPACED  MULTIPLE 
TARGETS,  U.  Nickel  (Forschungsinstitut  fur  Funk  und  Mathematlk, 

5307  Wachtberg-Werthhoven,  F.R.  Germany).  This  paper  is  concerned 
with  the  directional  resolution  of  M radiating  sources,  being  not 
distinguishable  by  conventional  Fourier  or  monopulse  processing. 

No  a priori  information  about  the  sources  is  assumed  to  be  available. 

The  estimation  portion  of  the  problem  is  carried  out  by  maximum 
likelihood  estimation.  The  resulting  non-linear  minimization  prob- 
lem is  investigated  and  a strong  dependency  of  the  conditioning  on 
the  sources'  phase,  amplitude  and  position  is  found.  Therefore,  a 
stochastic  approximation  algorithm  for  the  minimum  search  seems 
more  appropriate  than  a minimization  based  on  a single  observation. 

The  hypothesis  testing  portion  of  the  problem  can  be  solved  by  a 
sequence  of  one-sided  tests  for  M «•  1 . . , M , For  each  M a de- 
cision can  be  made  either  by  using  knowledge  of  the  accuracy  of  the 
estimation  problem  at  known  SNR  or  by  using  the  likelihood  ratio 
when  the  noise  level  is  known. 


OPTIMAL  DETECTION  OF  A TWO- STATE  MARKOV  PROCESS  IN  NOISE,  Erdal 
Panayircl  (Technical  University  of  Istanbul,  Istanbul,  Turkey). 

In  this  paper,  the  problem  of  detecting  a two-state,  continuous- 
time  Markov  process  with  a fixed  observation  time  in  additive  white 
Gaussian  noise  is  investigated.  It  is  considered  in  both  the  dis- 
crete and  continuous  domains,  and  computationally  feasible  algorithms 
are  derived  which  optimally  solve  the  problem. 
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The  optimum  detector  is  defined  by  the  Bayes  decision  rule  and  its 
basic  components  are  generated  Iteratively.  It  is  shown  that  the 
optimal  decision  rule  is  an  element  of  a class  of  on-line  decision 
rules  with  fixed  memory.  However,  the  number  of  computations 
needed  to  make  the  optimum  decision  grows  linearly  with  length  of 
the  observation  sequence. 

In  order  to  analyze  the  quality  of  the  optimal  detector,  it  is 
necessary  to  consider  the  probabilities  of  errors  of  the  first  and 
second  kind.  For  this,  the  conditional  error  probabilities  are 
first  studied  and  iterative  relationships  established  for  them  from 
which  the  usual  probabilities  are  found  as  a function  of  the  ob- 
servation time  and  the  parameters  of  the  problem. 

Finally,  the  case  of  continuous  observation  times  is  considered. 
Stochastic  differential  equations  are  obtained  for  the  logarithm  of 
the  likelihood  ratio  and  for  the  conditional  probabilities  of 
errors.  Since  these  equations  are  non-linear  as  well  as  stochastic, 
one  must  not  expect  to  obtain  any  exact  analytical  solutions  for 
them.  However,  some  approximate  methods  are  given  at  the  end  of 
the  paper. 


DIGITAL  SIGNAL  DETECTION  WITH  QUANTIZED  OBSERVATIONS,  Sam  Relsenfeld 
(Department  of  Systems  Science,  University  of  California,  Los  Angeles, 
Los  Angeles,  CA  90024).  The  problem  considered  is  a binary,  base- 
band digital  communications  system.  The  continuous  time  additive 
white  Gaussian  noise  channel  model  is  assumed.  The  channel  output 
is  low  pass  filtered,  sampled,  quantized  and  put  through  a digital 
signal  processor.  The  digital  signal  processor  output  is  the  de- 
tected bit  stream.  L time  samples  are  used  in  each  bit  decision, 
and  the  quantizer  has  N levels.  The  digital  signal  processor  is 
ideal  in  the  sense  that  it  has  no  internal  quantization  error.  The 
quantizer  preceding  the  digital  signal  processor  Imposes  a funda- 
mental limitation  on  the  detection  performance,  and  this  limitation 
is  Investigated. 

The  performance  measure  which  is  used  is  Pg,  the  probability  of  de- 
cision error.  The  Pg  performance  is  Investigated  as  a function  of 
Eb/Ng,  L,  and  N.  For  N ■ 2,  or  binary  quantization,  it  is  shown 
that. 


1.  u.  b.  P • 11m  P » erfc( 
e e 
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Therefore,  binary  quantization  of  the  channel  output  implies  a 
maximum  performance  loss  of  tt/2  or  1.961198771  dB. 
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For  N greater  than  2,  the  joint  distribution  of  the  number  of  times 
each  quantizer  output  is  reached  in  one  bit  Interval  is  multinomial. 
The  optimal  signal  detection  algorithm  is  derived  for  all  L and  N. 
The  special  case  asymptotic  form  of  this  algorithm  as  L or  N becomes 
large  is  Investigated.  The  asymptotic  Pg  performance  of  the  optimal 
detector  as  L becomes  large  is  also  Investigated. 


DIGITAL  DETECTION  OF  PERIODIC  SIGNALS  IN  GAUSSIAN  NOISE,  Mario  A. 
Blanco  (Electrical  Engineering  Department,  Illinois  Institute  of 
Technology,  Chicago,  IL  60616).  The  problem  of  digital  detection 
of  periodic  signals  with  random  phase  in  additive  gaussian  noise  is 
considered.  An  interesting  class  of  digital  receivers  for  the  de- 
tection of  these  signals  based  on  hardllmitlng , sampling  and  corre- 
lation operations  is  presented.  The  optimization  of  these  receivers 
is  considered  and  it  is  shown  how  one  can  approximate  the  operation 
of  the  maximum  likelihood  detector  by  very  simple  correlation  opera- 
tions. An  analysis  of  the  statistical  performance  of  these  digital 
detectors  is  then  presented,  and  it  is  shown  that  under  some  sim- 
plifying assumptions  one  can  obtain  closed  form  expressions  for  the 
expected  value,  variance  and  distribution  function  of  the  output  of 
the  detector.  Some  of  these  results  are  then  extended  to  the  case 
in  which  the  signal  to  be  detected  has  random  phase,  and  random 
time  of  arrival.  Also,  some  examples  of  the  performance  of  these 
detectors  as  a function  of  signal-to-noise  ratio  and  length  of  ob- 
servation interval  are  presented. 


DETECTION  OF  WEAK  SIGNALS  IN  NARROWBAND  NON-GAUSSIAN  NOISE,  J.  W. 
Modestino  and  A.  Nlngo  (Electrical  and  Systems  Engineering  Depart- 
ment, Rensselaer  Polytechnic  Institute,  Troy,  NY  12181).  A class 
of  nonlinear  receiver  structures  is  described  for  the  detection  of 
weak  signals  in  non-Gaussian  narrowband  noise.  In  particular,  the 
concept  of  a locally  optimum  receiver  structure  is  extended  to  the 
case  of  narrowband  signal  and  noise  models.  A useful  class  of  non- 
Gaussian  narrowband  noise  models  is  developed  for  which  the  locally 
optimum  receiver  implementation  is  explicitly  determined.  These 
structures  are  shown  to  provide  considerable  improvement  over  con- 
ventional linear  receiver  structures.  The  basis  of  comparison  is 
taken  as  the  asymptotic  relative  efficiency  (A.R.E.).  Unfortunately, 
the  locally  optimum  receiver  requires  explicit  a priori  knowledge  of 
the  underlying  noise  distribution.  To  circumvent  this  difficulty  a 
rather  simple  adaptive  nonlinear  receiver  structure  is  described 
which  attempts  to  adapt  to  the  unknown  prevailing  noise  environment. 
This  adaptive  receiver  is  shown  to  provide  fairly  efficient  and 
robust  performance  in  a wide  variety  of  non-Gaussian  narrowband 
noise  environments. 
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LEADING  EDGE  ESTIMATION  ERRORS,  Israel  Bar-David  and  David  Anaton 
(Department  of  Electrical  Engineering,  Technion  - Israel  Institute 
of  Technology,  Haifa,  Israel).  The  leading-edge-estimator  (LEE)  of 
the  time-of-arrival  (TOA)  of  a pulse  signal  is  defined  as  the  in- 
stant at  which  the  noise-contaminated  and  filtered  version  of  the 
received  signal  crosses  a preset  threshold  level.  A rigorous  analy- 
sis, valid  for  rectangular  pulses  in  additive  white  Gaussian  noise. 
Indicates  that  the  probability  density  of  the  LEE  is  asymmetrical 
and  therefore  the  LEE  is  biased.  Furthermore,  for  a pre-speclf led 
probability  of  anomalous  estimation  error,  the  minimum  bias  is  in- 
versely proportional  to  the  signal-to-noise  ratio  R while  the  mini- 
mum variance  is  inversely  proportional  to  R^.  Results  are  presented 
in  the  form  of  parameterized  graphs. 


MULTI-MODAL  ESTIMATORS  OF  PULSE  PARAMETERS  FOR  THRESHOLD-EXTENSION 
DEMODULATORS,  I.  Bar-David  and  B.  G.  Goldberg  (Faculty  of  Electrical 
Engineering,  Technion  - Israel  Institute  of  Technology,  Haifa, 
Israel).  The  multi-modal  (MM)  estimator  of  a non-linear ly  involved 
signal  parameter  is  a generalization  of  the  maximum  likelihood  (ML) 
estimator  in  that  it  records  M relative  maxima,  and  the  correspond- 
ing modes  of  the  likelihood  function  associated  with  the  parameter. 
The  additional  modes  contain  valuable  information  about  the  para- 
meter in  cases  when  anomalous  noise- induced  maxima  override  the 
signal-induced  one.  A class  of  ad-hoc  algorithms  is  introduced 
that  exploits  this  information  as  well  as  the  assumed  smoothness  of 
the  message  that  underlies  the  sequence  of  transmitted  parameters, 
in  order  to  lower  the  signal-to-noise  ratio  at  which  the  noise- 
threshold  effect  occurs.  Monte-Carlo  simulation  of  a typical  al- 
gorithm applied  to  a 2-modal  estimator  of  speech  signal  exhibits  a 
2 to  3 dB  Improvement  in  demodulating  speech  and  R.C.  filtered 
noises,  as  expected  from  a plausibility  argument. 
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EXTENSION  OF  AN  ADAPTIVE  DISTRIBUTED  ROUTING  ALGORITHM  TO  MIXED 
MEDIA  NETWORKS,  Anthony  Ephremides  (Electrical  Engineering  Depart- 
ment, University  of  Maryland,  College  Park,  MD) . By  modeling  the 
broadcast  portion  of  a mixed  media  network  as  a fully  connected 
point-to-point  network  with  link  capacities  varying  as  functions 
of  the  traffic  rate,  it  is  possible  to  extend  an  adaptive  distri- 
buted routing  algorithm  that  was  originally  developed  for  point-to- 
point  ground  networks.  Additional  modifications  for  improved 
dynamic  performance  at  the  satellite  interface  message  processors 
are  also  Included. 


ANALYSIS  OF  AN  ADAPTIVE  ROUTING  STRATEGY  FOR  COMPUTER  COMMUNICATION 
NETWORKS,  Tak-Shing  Yum  and  Mischa  Schwartz  (Columbia  University, 

New  York,  NY).  Adaptive  routing  strategies  have  been  adopted  for 
such  store- and- forward  data  networks  as  the  US  ARPA  network  and  the 
Swedish  TIDAS  network.  They  have  been  studied,  primarily  with  the 
use  of  simulation,  by  many  investigators.  Analysis  of  adaptive 
routing  algorithms  for  networks,  involving  the  interactive  opera- 
tion of  networks  of  queues,  has  not  received  as  much  attention  be- 
cause of  its  complexity. 

In  this  paper  we  present  an  approximate  analysis  of  an  adaptive 
routing  strategy  that  is  related  to  a strategy  studied,  using  simu- 
lation, by  the  British  National  Physical  Laboratory,  The  strategy 
consists  of  sending  messages  via  the  shortest  path  route  if  it  is 
unique.  If  two  or  more  alternate  paths  with  the  same  number  of 
links  to  the  destination  node  exist,  the  one  with  the  shortest  out- 
going queue  at  the  current  link  is  chosen.  The  analysis  at  any  one 
node  Involves  a variant  of  the  "Join  the  Shortest  Queue"  problem 
and  has  been  carried  out  using  a Markov  chain  finite  buffer  approach. 
(The  variant  is  due  to  the  fact  that  the  traffic  into  any  queue  has 
both  a fixed  component,  and  a variable  component  due  to  the  shortest 
path  algorithm.) 

Simulation  has  Indicated  that  the  interdeparture  times  for  messages 
leaving  queues  are  closely  exponential.  This  makes  queues  at  dif- 
ferent nodes  effectively  independent,  except  for  a set  of  parameters 
coupling  the  flows  together  to  ensure  flow  conservation  holds.  These 
parameters  were  found  using  a relaxation  scheme. 

The  analysis  described  has  been  applied  to  an  example  four-node  net- 
work. Various  traffic  matrices  were  assumed,  both  symmetrical  and 
asymmetrical.  In  all  cases  this  adaptive  routing  strategy  resulted 
in  a substantially  lower  average  message  time  delay  than  the  best 
fixed  strategy  minimizing  average  time  delay.  The  analytic  results 
were  found  to  agree  with  simulation  results  to  within  2%. 
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' OPTIMAL  ROUTING  FOR  LINE-SWITCHED  DATA  NETWORKS  USING  DISTRIBUTED 

COMPUTATION,  Adrian  Segall  (Department  of  Electrical  Engineering, 
Technlon  - Israel  Institute  of  Technology,  Haifa,  Israel).  An  al- 
gorithm for  optimal  routing  in  line-switched  data-communlcation 
networks  is  presented.  The  algorithm  is  such  that  every  node  in 
the  network  uses  the  information  received  from  its  neighbors  to 
update  its  information  and  to  reroute  messages.  The  algorithm  has 
the  property  of  producing  at  every  instant  loop-free  routing,  and 
for  stationary  Inputs  and  fixed  topology,  it  reduces  the  average 
delay  in  the  network  at  every  step  and  converges  to  the  optimal 
routing.  In  addition,  it  has  the  important  property  of  not  starting 
a new  updating  step  before  the  previous  one  is  completed. 


PERFORMANCE  OF  QUEUEING  SYSTEMS,  M.  K.  Nguyen  (RCA,  Government  Com- 
munication System,  Camden,  NJ  08102)  and  R.  L.  Pickholtz  (George 
Washington  University,  Washington,  DC).  Queueing  problems  are  cen- 
tral to  various  aspects  of  data  networks.  Many  efforts  have  been 
spent  on  the  theory  of  queues  in  finding  explicit  analytical  solu- 
tions. These  solutions  are  often  too  complicated  to  be  practical. 
The  present  paper  is  Intended  to  give  some  inequalities  for  various 
quantities  of  interest  in  queueing  system  performance.  The  in- 
equalities for  queue  length  distribution  in  GI/G/1  have  been  esta- 
blished (in  both  transient  and  equilibrium  states).  An  application 
example  for  the  expected  queue  size  in  a bilk  queue  has  also  been 
derived.  Finally  the  inequalities  of  expe^  ted  queue  size  are  given 
in  the  case  of  stations  having  finite  K waiting  places. 


ON  A STOCHASTIC  INTEGRAL  EQUATION  MODEL  FOR  MARKOV  QUEUEING  NETWORKS, 
Frederick  J.  Beutler  (Computer,  Information  and  Control  Engineering 
Program,  The  University  of  Michigan,  Ann  Arbor,  MI  48109).  A set 
of  stochastic  integral  equations  is  presented  as  an  axiomatic  formu- 
lation for  a class  of  queueing  networks  such  as  might  be  encountered 
in  communication  or  computer  systems.  Certain  requirements  on  ar- 
rival and  departure  functions  assure  that  the  vector  of  queue  lengths 
at  the  server  nodes  is  well-defined  as  a Markov  process,  which  be- 
haves in  accordance  with  the  usual  notion  of  a queueing  network.  At 
the  same  time,  the  model  possesses  the  flexibility  to  represent 
queueing  parameters  such  as  blocking,  jockeying,  customer  losses, 
variable  service  rates,  and  multiple  servers  at  each  node. 

Necessity  conditions  for  equilibrium  are  derived  in  terms  of  the 
Integral  equations;  for  Jackson  networks,  these  coincide  with  the 
traffic  equation.  The  Infinltesmal  generator  for  the  Markov  pro- 
cess Is  related  to  statistical  equilibrium  and  the  balance  equation. 
The  Infinltesmal  generator  is  then  exhibited  In  explicit  form. 
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The  set  of  stochastic  Integral  equations  can  be  augmented  to  account 
for  customer  streams.  The  augmented  process  Is  also  Markov,  and  Its 
Inflnltesmal  generator  can  be  obtained  with  little  difficulty. 


THEORETICAL  UPPER  BOUNDS  ON  SPECTRAL-SPATIAL  UTILIZATION  IN  A CELLULAR 
LAND  MOBILE  COMMUNICATIONS  SYSTEM,  G.  R.  Cooper  and  R.  W.  Nettleton 
(School  of  Electrical  Engineering,  Purdue  University,  West  Lafayette, 
IN  47907).  This  short  paper  presents  theoretical  upper  bounds  on 
spectral-spatial  utilization  In  a cellular  land  mobile  communications 
system. 

Spectral-spatial  utilization  (SSU)  is  defined  for  the  purposes  of 
this  paper  to  be  the  maximum  number  of  similar  message  channels 
per  MHz  for  each  cell  of  the  system.  It  Is  assumed  that  an  analog 
message  such  as  speech  is  being  transmitted  on  each  channel,  and 
message  quality  is  held  constant  as  a parametric  criterion. 

Two  possible  spectral  configurations  are  considered: 

1.  The  use  of  orthogonal  channels,  e.g.,  frequency-division 
multiplex.  In  which  channels  are  re-used  on  several  cells 
separated  by  a suitable  geographic  distance. 

2.  The  use  of  non  orthogonal  channels,  e.g.,  spread-spectrum 
with  code-dlvi  Ion  multiplex.  In  which  each  channel  is 
permanently  assigned  to  a single  user. 

Both  these  configurations  are  Interference-limited  In  performance: 
the  first  by  a relatively  small  number  of  co-channel  sources  dis- 
tributed In  the  service  area,  and  the  second  by  large  numbers  of 
sources  uniformly  distributed  over  all  the  service  area. 

Under  simple  but  not  unreasonable  assumptions  on  the  nature  of  the 
service  area,  it  Is  found  that  the  bound  on  the  SSU  of  a non- 
orthogonal-channel  system  Is  always  significantly  higher  than  the 
bound  on  the  SSU  of  the  orthogonal-channel  system,  over  a wide 
range  of  message  quality  criteria  and  propagation  characteristics. 

The  authors  are  currently  Investigating  a spread-spectrum  Imple- 
mentation of  the  non-orthogonal  channel  concept. 


A PROTOCOL  FOR  RESOLVING  CONFLICTS  ON  ALOHA  CHANNELS,  John 
Capetanakls  (22  Blake  Street,  Cambridge,  MA  02140).  When  a number 
of  sources  have  Independent  access  to  a common  channel,  conflicts 
normally  arise  when  more  than  one  source  attempt  to  access  the  chan- 
nel simultaneously.  Two  algorithms  have  been  proposed  to  resolve 
these  conflicts,  the  TDMA  and  the  Aloha.  The  problem  with  these 
protocols,  however,  is  that  they  suffer  either  from  congestion  In- 
stabilities or  from  low  throughput  and  high  mean  delay. 
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We  have  developed  a class  of  multiple  access  tree  search  algorithms 
that  do  not  have  the  above  difficulties.  We  will  present  these  al- 
gorithms and  show  that  they  are  stable,  have  a throughput  of  .43 
packets/slot,  and  have  superior  delay  characteristics.  Furthermore, 
we  will  discuss  their  optimality  and  show  that  TDMA  Is  a special 
case  of  the  tree  algorithms. 


INTEGRATED  RANDOM-ACCESS  RESERVATION  SCHEMES  FOR  MULTI-ACCESS  COM- 
MUNICATION CHANNELS,  Izhak  Rubin  (Department  of  System  Science, 

School  of  Engineering  and  Applied  Science,  University  of  California, 
Los  Angeles,  CA  90024).  Dynamic  demand-assignment  schemes,  govern- 
ing the  access-control  discipline  for  a network  of  terminals  com- 
municating through  a multi-access  communications  channel,  are  pre- 
sented. A repeater  Is  employed  to  allow  a fully-connected  network 
structure.  The  channel  can  be  characterized  as  Inducing  a low 
propagation-delay  value,  as  for  terrestrial  radio  or  line  networks; 
or  as  being  associated  with  a higher  propagation-delay  value,  as 
for  a satellite  communication  channel.  A synchronized  (slotted) 
communication  medium  Is  considered. 

Recent  results  we  have  derived  for  the  delay-throughput  performance 
of  dynamic  reservation  and  group  random-access  access-control  schemes 
are  presented.  Integrated  random-access  reservation  schemes  are  then 
presented  and  studied.  These  schemes  adapt  their  protocol  to  the 
underlying  traffic  load  using  combined  random-access  and  reservation 
access-control  procedures.  Such  procedures  are  subsequently  shown 
to  yield  excellent  delay-throughput  performance  characteristics  over 
the  whole  range  of  network  traffic  Intensity  values. 


MULTIPLICATIVE  MULTIPLE -ACCESS  METHOD  FOR  THE  INQUIRY/RESPONSE 
CHANNEL,  J.P.S.  Bhullar  and  J.  B.  Anderson  (Department  of  Electrical 
Engineering  and  Communications  Research  Laboratory,  McMaster  Uni- 
versity, Hamilton,  Ontario,  Canada,  L8S  4L7).  In  an  Inquiry /response 
channel,  very  many  users  send  Infrequent,  short  messages,  and  address- 
ing Information  makes  up  a major  component  of  the  Information  flow. 

We  propose  a scheme  to  accomplish  optimal  multiple  access  for  such  a 
channel  when  the  channel  Is  a digital  loop,  and  give  a simple  Imple- 
mentation. More  than  one  user  may  successfully  signal  at  once; 
without  this  feature,  a system  suffers  either  a loss  of  efficiency 
or  a throughput  delay.  In  the  scheme's  operation,  each  user  in  turn 
views  a single  circulating  Information  word  as  it  passes  through 
him.  If  he  wishes  to  signal  a (short)  message,  he  multiplies  the 
word  by  one  of  a collection  of  words  identified  only  with  him.  For 
Information  flowing  the  opposite  way,  the  user  checks  to  see  If  one 
of  his  words  Is  a factor  of  the  circulating  word.  For  1-blt  mes- 
sages, the  prime  numbers  provide  a good  source  of  codes.  By  the 
prime  number  theorem,  W words  can  be  signalled  up  to  N at  a time 
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in  a N log  W bit  word.  Conversely,  it  can  be  shown  that  the  mes- 
sage, addressing,  and  routing  information  requirement  tends  to  about 
this  many  bits  for  small  N.  However,  simpler  schemes  than  our 
multiplication  method  exist  which  do  this  well  for  small  N,  so  we 
stress  Instead  the  case  for  large  N.  Results  are  presented  of 
searches  for  large-N  codes,  not  necessarily  based  on  primes,  which 
almost  always  have  an  unambiguous  decoding.  Applications  to  polling 
and  to  the  distribution  of  short  messages  are  discussed.  Encoder 
and  decoder  hardware  requires  a multiplier,  divider,  and  a table 
look-up  only. 
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NEW  BOUNDS  TO  R(D)  OF  A SOURCE  WHOSE  OUTPUT  IS  THE  SUM  OF  TWO  IN- 
DEPENDENT RANDOM  ENTITIES,  Dimitris  Anastasslou  and  David  J. 

Sakrison  (Department  of  Electrical  Engineering  and  Computer  Sciences 
and  the  Electronics  Research  Laboratory,  University  of  California, 
Berkeley,  CA  9A720).  In  an  effort  to  apply  results  of  Information 
theory  to  the  efficient  storage  or  transmission  of  images,  it  has 
been  found  that  the  intensity  random  field  of  a typical  image  could 
be  represented  as  the  sum  of  two  independent  random  fields  of  sim- 
pler description;  it  has  also  been  found  that,  under  some  conditions, 
the  frequency-weighted  squared-error  criterion  is  a satisfactory 
model  for  the  fidelity  evaluation  of  the  images.  Thus,  a situation 
of  practical  interest  is  when  the  source  output  is  the  sum  of  two 
independent  random  entities,  whose  corresponding  rate-distortion 
functions  with  respect  to  the  (probably  frequency-weighted)  squared- 
error  criterion,  can  be  found  or  approximated.  The  values  of  these 
functions  have  been  used  to  provide  new  bounds  to  the  rate-distor- 
tion function  of  the  additive  source  with  respect  to  the  same  cri- 
terion. These  bounds  have  been  derived  by  use  of  either  special 
encoding  strategies  or  conditional  rate-distortion  theory;  some  of 
them  are  shown  to  be  tighter  than  the  known  bounds  for  certain  re- 
gions of  distortion,  and  have  a desirable  asymptotic  behavior  for 
small  distortion  levels. 


RATE-DISTORTION  FUNCTIONS  FOR  CONTINUOUS  ALPHABET  MEMORYLESS  SOURCES, 
Stephen  L.  Fix  and  David  L.  Neuhoff  (University  of  Michigan,  Ann  Arbor, 
MI  A8109).  The  analytical  evaluation  of  the  rate-distortion  func- 
tion for  memoryless  sources  with  continuous  alphabets  has  been  until 
recently  limited  to  Gaussian  sources  and  the  squared  error  distortion 
criterion  and  cases  where  the  Shannon  lower  bound  is  tight.  Recently 
Tan  and  Yao  have  calculated  the  rate-distortion  function  for  a 
Gaussian  source  with  a magnitude-error  distortion  criterion  and  have 
shown  that  their  procedure  can  be  applied  to  similar  sources.  This 
talk  deals  with  various  analytical  techniques  available  in  finding 
rate-distortion  functions.  Specifically  the  squared-error  distor- 
tion criterion  will  be  discussed. 

It  has  been  thought  that  Blahut's  algorithm  would  be  an  efficient  way 
to  compute  R(D)  for  continuous  alphabet  sources.  However,  as  will 
be  discussed,  for  most  sources  and  the  squared-error  distortion  cri- 
terion the  solution  will  be  discrete  and  therefore  the  algorithm  will 
need  modification  to  work  efficiently.  A modified  algorithm  is  sug- 
gested and  examples  discussed. 


ON  THE  ASYMPTOTIC  BEHAVIOUR  OF  THE  RATE -DISTORT I ON  FUNCTION,  Ge'rard 
Cohen  (ENST,  A6  rue  Barrault,  75013  Paris,  France)  and  Claudia  Lidia 
Sinloneacu  (Department  of  Mathematics,  University  of  Brasov,  2200 
Brasov-R.S.  Romania).  The  paper  deals  with  a result  on  the  asymptotic 
behaviour  of  the  rate-distortion  function  for  a quite  general  class 
of  source  distributions  and  difference  distortion  measures. 
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AN  APPROXIMATION  TO  RATE-DISTORTION , Gerard  Cohen  (Ecole  Nationale 
Superleure  des  PTT,  A6  Rue  Barrault,  75013  Paris,  France)  and 
Claudia  Lidia  Simionescu  (Department  of  Mathematics,  University  of 
Brasov,  2200  Brasov-R.S.  Romania).  The  paper  deals  with  a new  ap- 
proach to  the  problem  of  computing  the  rate-distortion  function  as- 
sociated with  a given  source  and  a given  fidelity  criterion.  For 
any  6>0,  we  construct  a minimal  set  of  conditional  probabilities 
Aq  - {Ql). 

Minimization  over  Aq  gives  an  approximation  to  the  rate-distortion 
function.  A convergent  algorithm  for  getting  minimal  sets  Aq  is 
shown  and  implementation  for  the  results  has  been  done. 

The  method  admits  generalizations  to  continuous  sources  and  more 
general  distortion  measures. 


BOUNDS  ON  THE  PERFORMANCE  OF  LINEAR  SOURCE  CODING,  T.  C.  Ancheta 
(IBM  Research,  Yorktown  Heights,  NY).  It  is  shown  that  when 
a binary  stationary  source  with  entropy  H is  transformed 
linearly  into  the  compressed  sequence  utilizing  R S H output  digits 
per  source  letter,  the  average  Hamming  distortion  in  reconstructing 
the  source  cannot  be  less  than  (1-R  )h"l  ( (H-Rj.)  / (1-R^) ) where  h is 
the  binary  entropy  function.  This  §ound  lies  above  the  rate-distor- 
tion function  of  the  binary  memoryless  source  (except  at  the  points 
Rc  “ {H,0}  where  they  meet).  For  the  binary  symmetric  source,  the 
bound  is  readily  shown  to  be  tight. 

We  consider  a universal  generalization  of  linear  source  coding  in 
which  the  source  sequences  are  partitioned  into  M subsets  and  a 
linear  mapping  is  associated  with  each  subset.  A source  sequence 
is  encoded  by  transmitting  a log2M-bit  prefix  to  indicate  the  subset 
to  which  the  sequence  be  longs  followed  by  the  compressed  digits  re- 
sulting from  the  mapping  corresponding  to  such  a subset.  It  is 
shown  that  the  performance  of  this  source  coding  scheme  in  com- 
pressing a stationary  source  is  bounded  aymptotlcally  by  the  same 
function  as  above  when  the  number  of  partitions  is  finite. 


JOINT  SOURCE-CHANNEL  TIME- INVARIANT  TRELLIS  ENCODING,  James  G. 

Dunham  and  Robert  M.  Gray  (Department  of  Electrical  Engineering, 
Stanford  University,  Stanford,  CA  94305).  One  of  the  fundamental 
problems  of  information  theory  is  the  Joint  source-channel  coding 
problem,  that  is,  the  optimal  transmission  of  a source  over  a noisy 
channel  to  a user  with  respect  to  a fidelity  criterion.  We  show  how 
the  theory  of  sliding-block  codes  can  be  used  to  obtain  a theory  of 
Joint  source-channel  time-invariant  trellis  encoding  for  discrete 
ergodlc  sources,  discrete  memory less  channels  and  bounded  single- 
letter  fidelity  criteria.  A block  trellis  encoder  Joint  source- 
channel  coding  theorem  is  stated  which  shows  that  the  Optimal 
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performance  Theoretically  Attainable  (OPTA)  for  these  codes  Is  given 
by  D(C),  the  distortion-rate  function  D(’)  evaluated  at  the  channel 
capacity  C.  This  generalizes  the  theory  of  source  time- invariant 
trellis  encoding  from  discrete  memoryless  noiseless  channels  to  dis- 
crete memoryless  noisy  channels.  Also,  this  leads  to  a new  method 
of  channel  coding  using  a per  symbol  error  criterion  and  these  new 
channel  codes  are  compared  with  time- invariant  convolutional  block 
channel  codes. 


TREE  SEQUENTIAL  ENCODING  FOR  SOURCES  WITH  MEMORY  UNDER  A FIDELITY 
CRITERION,  Harry  H.  Tan  (Department  of  Electrical  Engineering  and 
Computer  Science,  Princeton  University,  Princeton,  NJ  08540).  A 
sequential  algorithm  for  encoding  the  outputs  of  discrete  time 
stationary  block  ergodlc  sources  under  a single  letter  fidelity 
criterion  using  tree  codes  is  analyzed.  It  is  shown  that  for  en- 
coding these  source-user  pairs,  there  are  tree  codes  that  satisfy 
the  rate-distortion  bound  with  the  algorithm  used  as  the  encoder. 
Moreover  the  average  number  of  computations  per  encoded  tree  branch 
required  by  the  algorithm  is  finite.  Bounds  on  computational  re- 
quirements are  also  developed. 


CAUSAL  RATE-DISTORTION  THEORY  FOR  REAL-TIME  ESTIMATION,  Julian  L. 
Center,  Jr.  (The  Analytic  Sciences  Corporation,  Six  Jacob  Way, 
Reading,  MA  01867).  Real-time  estimation  requires  a causal  esti- 
mation scheme;  that  is,  future  measurements  cannot  be  used  to  con- 
struct current  estimates.  This  paper  extends  rate-distortion 
theory  to  include  causality  constraints.  A formulation  of  the 
rate-distortion  problem  with  causality  constraints  Is  presented, 
and  the  character  of  the  solution  to  this  problem  is  discussed. 

For  the  linear-system,  quadratic  distortion,  Gausslan-nolse  case, 
a technique  yielding  approximate  solutions  is  presented.  This 
technique  leads  directly  to  an  easily- realizable  system  operating 
on  the  approximate  rate-distortion  boundary. 
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OPTIMIZING  RECEIVERS  FOR  REMOTE  PERCEPTION  USING  SENSORY  NOISE  RE- 
DUCTION TECHNIQUES,  Charles  Honorton  (Department  of  Psychiatry, 
Mairaonides  Medical  Center,  Brooklyn,  NY).  We  report  a series  of 
experiments  demonstrating  remote  information  transfer  between  a 
source  (human  sender  + target  message)  and  a sensor ially-isolated 
human  receiver  in  another  room,  4.5  m distant.  Receiver  optimiza- 
tion techniques  were  used  to  enhance  detection  and  retrieval  of  re- 
mote stimuli  by  attenuating  patterned  perceptual  input  and  pro- 
prioception, considered  as  likely  noise  sources  in  remote  percep- 
tion. The  generalizability  of  these  findings  is  supported  by  in- 
dependent replications  in  six  different  laboratories.  A film 
demonstrating  an  experimental  session  will  be  shown  preceding  this 
talk. 


AXIOMATIZATION  AND  REPRESENTATION  OF  QUALITATIVE  INFORMATION, 

Zoltan  Domotor  (University  of  Pennsylvania).  Given  a Boolean  field 
of  measurable  sets  impressed  with  a binary  comparative  informativeness 
relation  and  a binary  informational  Independence  relation,  necessary 
and  sufficient  conditions  are  provided  for  the  existence  of  a Shannon 
information  measure  on  the  field  such  that  it  respects  the  indepen- 
dence and  agrees  with  the  comparative  informativeness.  The  relevant 
axioms  are  tracked  down  by  solving  finite  systems  of  bilinear  in- 
equalities in  symmetric  product  spaces. 

A representation  using  non-Archlmedean  information  measures  is  pre- 
sented, based  on  the  compactness  property  of  finite  substructures. 

The  notion  of  information-preserving  map  is  introduced  for  discuss- 
ing some  of  the  basic  constructions,  such  as  quotients,  products, 
and  filtered  limits  of  information  spaces. 

Arguments  are  displayed  in  favor  of  qualitative  information  theory 
and  some  directions  are  suggested  for  future  research. 


INFORMATION  THEORY  AND  ORGANIZATION  THEORY,  R.  F.  Drenick,  (Poly- 
technic Institute  of  NY,  333  Jay  Street,  Brooklyn,  NY  11201).  The 
paper  reports  on  a recent  mathematical  approach  to  organization 
theory  which  relies  heavily  on  concepts  drawn  from  information  theory. 
These  concepts  were  found  to  be  useful  in  the  modelling  of  the  in- 
dividual organization  member,  as  well  as  in  the  characterization  of 
the  functioning  of  the  organization  as  a whole.  The  model  of  the 
member  in  particular  is  based  on  Shannon's  original  definition  of  a 
noiseless  channel.  Extensions  however  were  Indicated  in  several 
directions,  partly  in  order  to  accommodate  observational  evidence, 
and  partly  in  order  to  define  operations  in  the  presence  of  "over- 
load" which  is  not  usually  considered  in  information  theory.  It  is 
shown  that  the  desire  to  avoid  overload  in  fact  accounts  for  many 
of  the  most  conspicuous  features  of  organizations  in  practice.  Con- 
cepts drawn  from  Information  theory  can  be  used  to  describe  their 
purpose  and  their  performance. 
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RECENT  WORK  ON  ALGORITHMIC  INFORMATION  THEORY,  Gregory  J.  Chaitln 
(IBM  Research,  Yorktown  Heights,  NY  10598).  Algorithmic  informa- 
tion theory  is  an  attempt  to  apply  information-theoretic  and  proba- 
bilistic ideas  to  recursive  function  theory.  Typical  concerns  in 
this  approach  are,  for  example,  the  number  of  bits  of  information 
required  to  specify  an  algorithm,  or  the  probability  that  a program 
whose  bits  are  chosen  by  coin  flipping  produces  a given  output. 
During  the  past  few  years  the  definitions  of  algorithmic  information 
theory  have  been  reformulated.  We  review  some  of  the  recent  work  in 
this  area. 


COMPLEXITY  BASED  INDUCTION  SYSTEMS:  COMPARISONS  AND  CONVERGENCE 
THEOREMS,  R.  J.  Solomonoff  (Rockford  Research,  Inc.,  140  1/2  Mt. 
Auburn  Street,  Cambridge,  MA  02138).  In  1964  we  proposed  as  an 
explication  of  a priori  probability,  the  probability  measure  in- 
duced on  output  strings  by  a universal  Turing  machine  with  uni- 
directional output  tape  and  a randomly  coded  unidirectional  input 
tape. 

Levin  has  shown  that  if  is  the  unnormallzed  form  of  this 

measure,  and  P(x)  is  any  computable  probability  measure  on  strings, 

X,  then 

P|;,(x)  2 C P(x) 

C being  a constant  Independent  of  x.  We  show  that  this  result  for 
Pf],  the  normalized  form  of  this  measure,  is  directly  derivable  from 
Willis'  probability  measures  on  non-universal  machines. 

If  the  conditional  probabilities  of  Pjlj  are  used  to  approximate  those 
of  P,  then  the  expected  value  of  the  total  squared  error  in  these 
conditional  probabilities  is  bounded  by  -1/2  In  C.  In  this  error 
criterion,  and  when  used  as  the  basis  of  a universal  gambling  scheme. 
Pill  is  superior  to  Cover's  measure,  b*. 

When  H*  • -log2  PjJj  is  used  to  define  the  entropy  of  finite  sequences, 
the  equation 

H*(x,y)  “ H*(x)  + H*(y) 

holds  exactly,  in  contrast  to  Chaltin's  entropy  definition,  which 
has  a non-vanishing  error  term  in  this  equation. 
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SYNDROME  DECODING  OF  BINARY  RATE-k/n  CONVOLUTIONAL  CODES,  J.  P.  M. 
Schalkwijk,  A.  J.  Vlnck  and  K.  A.  Post  (Eindhoven  University  of 
Technology,  Eindhoven,  The  Netherlands).  This  paper  concerns  a 
state  space  approach  to  syndrome  decoding  of  binary  rate-k/n  con- 
volutional codes.  State  space  symmetries  of  a certain  class  of 
codes  can  be  exploited  to  obtain  an  exponential  reduction  of  the 
hardware  of  a maximum  likelihood  decoder.  Simulation  results  in- 
dicate that  the  state  space  symmetries  can  also  be  used  to  advantage 
in  sequential  decoding.  In  Fano  and  stack  decoding,  one  obtains 
major  savings  in  the  number  of  computations  and  in  the  required 
storage.  Aside  from  these  practical  advantages,  i’  is  felt  that 
the  state  space  formalism  developed  in  this  paper  has  some  intrinsic 
value  of  its  own. 


CONCATENATED  CODES  FOR  IMPROVED  PERFORMANCE  WITH  APPLICATIONS  TO 
THL  RAYLEIGH  FADING  CHANNEL,  John  F.  Pieper  (Stein  Associates, 
Waltham,  MA) , John  G.  Proakis  (Northeastern  University,  Boston,  MA) , 
Roger  R.  Reed  (Stein  Associates,  Waltham,  MA) , and  Jack  K.  Wolf 
(University  of  Massachusetts,  Amherst,  MA) . The  concatenation  of 
two  block  codes  is  an  effective  procedure  for  generating  a block 
code  of  very  large  size.  A specific  form  of  concatenation  and  the 
associated  decoding  algorithm  are  briefly  discussed.  Performance 
' results  for  a Rayleigh  fading  channel  are  presented  which  show  that 

a savings  of  several  dB  is  obtainable  in  this  manner. 


CONTINUED  FRACTIONS  AND  BERLEKAMP'S  ALGORITHM,  L.  R.  Welch  and 
R.  A.  Scholtz  (Department  of  Electrical  Engineering,  University  of 
Southern  California,  Los  Angeles,  CA  90007).  The  problem  of  deter- 
mining the  error-locator  polynomial  and  error-evaluator  polynomial 
in  decoding  BCH  codes  is  equivalent  to  determining  a ratio  of  poly- 
nomials from  a given  number  of  coefficients  in  its  formal  power 
series  expansion.  Viewing  polynomials  as  a ring  of  "integers" 
within  a field  of  formal  power  series,  the  decoding  problem  can  be 
translated  into  one  of  finding  rational  approximations  to  field 
elements . 

Continued  fractions  have  been  used  for  many  years  to  generate  good 
rational  approximations  to  real  numbers.  In  this  paper  the  theory 
of  continued  fraction  approximations  is  developed  abstractly  for 
ring-field  combinations  on  which  a non-archlraedean  norm  is  defined 
and  for  which  the  "integer-part"  mapping  from  field  elements  to 
ring  elements  satisfies  two  simple  postulates.  Basic  theorems  con- 
cerning the  optimality  of  continued  fraction  approximations  gener- 
ated in  this  setting  are  given  and  the  connection  with  the  BCH 
decoding  problem  la  established. 
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A DECODING  PROCEDURE  FOR  MULTIPLE-ERROR-CORRECTING  CYCLIC  AN  CODES, 
Wen-Yung  Yeh  (Department  of  Electrical  Engineering,  Northeastern 
University,  Boston,  MA  02115).  A decoding  procedure  for  multiple- 
error-correcting  arithmetic  cyclic  codes  is  proposed.  For  cyclic 
AN  codes  Cj.(n,B,t)  with  information  rate  logj-B/n  less  than  1/t,  the 
error  trapping  technique  will  correct  all  correctable  error  patterns 
If  the  information  rate  is  larger  than  1/t  but  less  than  2/t,  a modi 
fled  error-trapping  decoding  scheme  has  been  developed.  Fortunately 
it  turns  out  that  all  useful  multiple-error-correcting  cyclic  AN 
codes  with  block  length  n less  than  58,  listed  in  the  appendix  B of 
previous  thesis  by  Larson,  can  be  decoded  effectively  by  this 
algorithm. 


ARITHMETIC  CODES  IN  RESIDUE  NUMBER  SYSTEMS,  Ferruccio  Barsl  and 
Piero  Maestrini  (Istltuto  dl  Elaborazlone  dell ' Inf ormazlone  del  CNR, 
Pisa,  Italy).  Residue  Number  Systems  (RNS)  provide  a means  to  con- 
struct non-binary,  multiple  error  correcting  arithmetic  codes.  In 
this  paper  two  different  codes,  namely  systematic  codes  in  RNS  and 
AN  codes  in  RNS  are  considered  and  their  properties  are  discussed 
in  detail.  A lower  bound  to  redundancy  allowing  t-correction  in  a 
class  of  codes  including  those  under  consideration  is  reported  and 
It  is  shown  that  systematic  codes  and  AN  codes  in  RNS  reach  this 
bound.  In  both  codes,  error  correction  is  achieved  by  finding  ap- 
propriate solutions  to  a key  congruence.  Two  different  decoding 
procedures  are  presented.  The  first  one,  although  working  with  any 
t-correcting  code,  is  computationally  inefficient  for  high  values 
of  t.  The  second  procedure,  based  upon  the  euclidean  algorithm  for 
integers,  is  very  efficient  but  requires  an  amount  of  redundancy 
slightly  above  the  lower  bound.  The  error  correcting  procedure 
based  upon  the  euclidean  algorithm  was  previously  known  for  AN  codes 
in  RNS.  In  the  case  of  systematic  codes  in  RNS,  this  procedure  is 
faster  than  the  best  previously  known  procedure,  requiring  multiple 
iterations  of  an  algorithm  based  on  convergence  of  continued 
fractions . 


131 


SESSION  G2 


A NOTE  ON  THE  IDENTIFICATION  OF  TWO-DIMENSIONAL  ARMA  MODELS,  H.  W. 
Penm  and  M.  Kanefsky  (University  of  Pittsburgh).  Recent  work  has 
focused  on  modeling  and  enhancement  of  two-dimensional  monochromatic 
images.  A major  complication  in  extending  one-dimensional  tech- 
niques is  the  fact  that  finite  order  factorization  of  the  spectrum 
is  usually  not  possible.  Woods  has  made  progress  in  the 
factorization  problem  but  truncation  methods  are  necessary  for 
finite  recursive  filtering  techniques.  Strlntzls  assumes  that  the 
power  spectrum  itself  can  be  modeled  by  finite  factors  or 
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where  N^^  i M^^,  N2  2 M2. 

This  two-dimensional  (N^^,  N2;  M^^,  M ) ARMA  model  results  in  optimal 
finite  recursive  filters  which  require  the  identification  of  the  A., 
or  AR  coefficients. 


A procedure  for  identifying  this  two-dimensional  ARMA  model  (l.e., 
both  the  order  and  the  AR  coefficients)  from  the  measured  spatial 
correlation  matrix  is  presented.  This  procedure  is  a modification 
of  the  one-dimensional  technique  of  Graupe,  Krause  and  Moore,  and 
is  easily  extended  to  N-dlmensional  data. 

While  the  ARMA  model  does  not  permit  simulation  of  data,  effects  of 
measurement  inaccuracies  can  still  be  analyzed.  This  can  be  accom- 
plished by  simulating  measured  spatial  correlation  function  directly. 
Using  such  procedures,  the  effects  of  measurement  error  on  an  example 
were  investigated  and  no  unexpected  instability  problems  were  evident 


THE  RECURSIVE  IDENTIFICATION  OF  STOCHASTIC  SYSTEMS  USING  AN  AUTOMATON 
WITH  SLOWLY  GROWING  MEMORY,  B.  D.  Kurtz  (Howden  Applied  Research, 
Toronto,  Canada)  and  P.  E.  Caines  (Division  of  Applied  Sciences, 
Harvard  University,  Cambridge,  MA) . Consider  a discrete  time  system 
or  process  with  l.i.d.  outputs,  Xj^,X2,...  The  distribution  of  X^'s 
is  characterized  by  some  unknown  parameter,  9,  which  is  an  element  of 
Z^,  the  positive  integers.  We  describe  an  algorithm  (automaton) 
wiiich  possesses  a slowly  growing  memory  and  which  generates  estimates 
converging  to  0w.p.l. 

The  following  three  specific  applications  are  discussed: 

1)  Identification  of  the  most  probable  value  of  a single 
sample  from  a set  of  discrete  l.i.d.  random  variables. 

2)  Identification  of  the  parameter  of  an  observed  sequence 
of  l.i.d.  Bernoulli  random  variables. 
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3)  Identification  of  the  parameters  of  a moving  average 
process.  (Notice  that  the  observations  of  the  system 
output  In  this  case  are  not  l.l.d.). 

An  estimate  of  the  rate  of  convergence  of  the  algorithm  exceeds  that 
of  other  proposed  recursive  algorithms  for  parameter  Identification. 
Furthermore,  the  amount  of  memory  required  Is  shown  to  grow  as  log 
log  1,  a rate  significantly  slower  than  that  required  by  other  al- 
gorithms. It  Is  also  shown  that  for  a very  large  class  of  systems, 
convergence  of  parameter  estimates  Is  Impossible  without  access  to 
an  unbounded  memory. 

Simulation  experiments  are  described. 


MLE  OF  DENSITY  FORMS  AND  IDENTITY  OF  NORMAL  MIXTURES,  D.  S.  Arantes 
(Unlversidade  Estadual  de  Campinas,  Calxa  Postal  1170,  Campinas  - 
S.P.  - Brazil).  Given  a set  of  l.l.d.  univariate  random  variables 
Xj^,  x-,  X.,  ...,  x^  distributed  according  to  f(x),  the  problem  is 
to  estimate  the  density  f(x)  or  a set  of  parameters©  characterizing 
f(x).  Strong  convergence  in  Information  (divergence)  is  established 
for  maximum  likelihood  estimates  of  densities  under  hypotheses  simi- 
lar to,  but  distinct  from,  those  of  Wald.  The  results  are  then  ap- 
plied to  the  problem  of  estimating  the  parameters  characterizing  a 
family  of  countable  mixtures  of  normal  densities. 


CONSISTENT  ESTIMATION  OF  THE  ORDER  OF  AUTOREGRESSIVE  SYSTEMS, 
William  G.  Hwang  (School  of  Electrical  Engineering,  Cornell  Uni- 
versity). Hitherto  suggested  estimators  of  system  order  have  not 
been  consistent.  We  propose  a statistically  consistent,  but  com- 
putationally demanding,  estimator  of  the  order  of  an  autoregressive 
process.  We  also  present  results  on  the  efficiency  of  our  method 
and  the  best  possible  efficiency  of  any  estimator  of  system  order. 


RESOLUTION  ENHANCEMENT  IN  THE  AUTOREGRESSIVE  SPECTRAL  ESTIMATION  OF 
NOISY  SIGNALS,  M.  Kaveh  (Department  of  Electrical  Engineering,  Uni- 
versity of  Minnesota,  Minneapolis,  MN  55455).  The  autoregressive 
(AR)  method  is  becoming  a preferred  mode  of  spectral  estimation.  A 
main  advantage  of  this  technique  Is  that  It  is  data-adaptlve  and 
free  from  the  effects  of  window  functions  associated  with  the  tradi- 
tional Blackman  and  Tukey  type  methods.  A main  weakness  of  the  AR 
spectral  estimation  method  is  the  strong  dependence  of  its  resolu- 
tion on  the  slgnal-to-nolse  ratio  of  the  data. 
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In  this  paper  a modified  autoregressive  model  based  on  the  auto- 
correlation samples  for  lags  beyond  the  Inverse  noise  bandwidth  Is 
followed  by  an  all-zero  model  estimated  from  the  autocorrelation 
samples  in  the  Inverse  of  the  noise  bandwidth.  For  small  signal- 
to-nolse  ratios  this  model  requires  many  fewer  parameters  than  the 
pure  autoregressive  one  In  estimating  the  power  spectrum  of  signals 
with  very  high  resolution.  Some  examples  are  shown  that  compare 
this  method  with  the  pure  AR  and  the  optimum  autoregresslve-movlng 
average  spectral  estimators. 
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PSEUDO-RANDOM  NUMBERS,  H.  G.  Niederrelter  (Mathematics  Department, 
University  of  Illinois,  Urbana,  IL  61801).  Pseudo-random  numbers 
are  used  for  the  deterministic  simulation  of  random  variables  and 
random  processes.  The  most  commonly  employed  pseudo-random  numbers 
for  simulating  the  uniform  distribution  are  those  generated  by  the 
linear  congruential  method.  The  main  part  of  the  talk  will  be 
devoted  to  recent  progress  concerning  the  statistical  properties 
of  such  pseudo-random  numbers. 

The  performance  of  linear  congruential  pseudo-random  numbers  under 
a variety  of  statistical  tests  has  been  investigated.  An  important 
requirement  for  the  applicability  of  pseudo-random  numbers  in  Monte 
Carlo  calculations  is  statistical  independence  of  successive 
terms.  Effective  results  on  the  deviation  of  linear  congruential 
pseudo-random  numbers  from  an  ideal  independence  behavior  are  now 
available.  The  results  lead  to  criteria  for  the  optimum  choice 
of  parameters  in  the  linear  congruential  method  in  order  to 
produce  a desired  Independence  property.  Information  about 
pseudo-random  numbers  generated  by  feedback  shift  registers  will 
also  be  given. 


SOME  ASPECTS  OF  CONVEXITY  THAT  IMPACT  INFORMATION  THEORY,  Hans 
Wltsenhausen  (Bell  Telephone  Laboratories,  Murray  Hill,  NJ  07974). 
From  its  very  beginning,  information  theory  was  pervaded  by  convexity 
arguments.  Much  of  the  necessary  background  was  developed  on  an  ad 
hoc  basis  without  reference  to  the  knowledge  existing  in  the 
mathematical  study  of  convex  sets  and  functions.  Yet,  explicit  use 
of  this  knowledge  is  helpful  in  several  areas  of  Information  theory, 
as  will  be  shown  by  examples. 


RECENT  RESULTS  IN  ERGODIC  THEORY,  Paul  C.  Shields,  (Mathematics 
Department,  University  of  Toledo,  Toledo,  OH  43606).  Several 
equivalent  ways  of  deciding  whether  or  not  a stochastic  process 
is  a sliding-block  coding  of  an  1.1. d.  process  will  be  described. 

In  the  course  of  these  descriptions  such  concepts  as  the  d -distance 
between  processes,  Rohlin  stacks,  and  the  Ornstein  isomorphism 
theorem  will  be  discussed. 
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MARKOV  RANDOM  FIELDS,  Frank  L.  Spltzer  (Mathematics  Department,  Cornell 
University,  Ithaca,  NY  14853).  Markov  random  fields  are  a special  class 
of  binary  stationary  stochastic  processes  whose  parameters  take  values  in 
Z,,  the  d-dimenslonal  lattice  points.  Discussion  will  focus  on  four 
equivalent  ways  of  characterizing  a Markov  random  field,  namely: 


1)  in  terms  of  conditional  probabilities  depending  only  on  nearest 
neighbor 

2)  as  a Gibbs  state  with  nearest  neighbor  potential 

3)  as  a t^me  reversible  equilibrium  state  of  certain  time  evolutions 
with  nearest  neighbor  interaction 

4)  as  a state  with  lowest  free  energy  or  highest  entropy,  among  all 
stationary  states  with  the  same  given  specific  energy. 


136 


AUTHOR  INVEX 


Author 

Page 

Number 

Author 

Page 

Number 

Abut,  H 

113 

Berger,  T. 

34 

Aczel,  J.  D. 

77 

Berlekamp,  E.R. 

88 

Adleman,  L. 

41 

Beutler,  F.  J. 

121 

Adoul,  P. 

68 

Bhargava,  V.  K. 

23 

Anastasslou,  D. 

125 

Bhullar,  J.  P.  S. 

123 

Anaton,  D. 

119 

Biglieri,  E. 

97 

Ancheta,  T.  C. 

126 

Blahut,  R.  E. 

109 

Anderson,  R. 

101 

Blanco,  H.  A. 

118 

Anderson,  J.  B. 

67,104,123 

Block.  H.  U. 

85 

Annecke,  K.  11. 

79 

Bobillot,  G. 

8b 

Arantes,  D.  S. 

133 

Boekee,  D.  E. 

65 

Arcese,  A. 

94 

Boxma , Y . 

65 

Arimoto,  S. 

74 

Brehm,  H. 

105 

Au,  S. 

31 

Breton,  J.  R. 

81 

Bachman , D . F . 

93 

Burnett,  J.  W. 

32 

Bahl , L. 

43 

Byrd,  R.  H. 

110 

Baker,  C.  R. 

74 

Cain,  J.  B. 

69 

Baras,  J.  S. 

50 

Caines,  P.  E. 

132 

Bar-David,  I. 

119 

Cambanls,  S. 

75 

Barduccl,  M. 

114 

Cantonl,  A. 

101 

Barsl,  F. 

131 

Capetanakis,  J. 

!'>? 

Beard,  J.  X. 

93 

Cappellinl,  V. 

114 

Benelll,  G. 

114 

Carpenter,  D.D. 

101 

Benvenuto,  N 

52 

Center  Jr . , J.  L. 

127 

137 


AUTHOR  IHVEK 


Author 

Page 

Number 

Author 

Page 

Number 

Chaltin,  G.  J. 

129 

de  Figuieredo,  R. 

83 

Chan,  V.  W.  S. 

52 

Del  Re.  E. 

114 

Chang,  N.  U. 

34 

Deliarca,  J.  R, 

68 

Chang , S . C . 

36 

Devroye,  L.  P. 

28 

Chase,  D. 

86 

Deza,  M. 

96 

Chen,  C.  L. 

22 

Dimltriadis,  B. 

27 

Chevillat,  P.  R. 

67 

Dobbins,  J. 

111 

Clark  Jr . , G.  C. 

69 

Domotor,  Z. 

128 

Claus,  A.  J. 

49 

Doralswami,  R. 

63 

Cohen,  G. 

96,125,126 

Dorsch,  B. 

78 

Cohn , D . L . 

58 

Drenick,  R.  F. 

128 

Cohn-Sfetcu,  S. 

94 

Due,  M,  Q. 

98 

Constantin,  S.  D. 

96 

Dunham,  J.  G. 

126 

Conway,  J.  U. 

22 

Dwye  r , R . F . 

47 

Cooper,  G.  R. 

122 

El  Gamal,  A. 

54 

Cordes,  P. 

39 

Ella,  M. 

97 

Costello,  D.  J. 

67 

Elias,  P. 

21 

Cot,  N. 

58 

El-Sawy,  A.  H. 

48 

Cover,  T.  M. 

21,54 

Ephremides,  A. 

120 

Cslszar,  1. 

107 

Evans,  R.  J. 

101 

UasGupta,  S. 

94 

Falconer,  D.  D. 

38 

Davidson,  F. 

32 

Fltingof,  B. 

59 

Davisson,  L.  D. 

90 

Fix,  S.  L. 

125 

Deaett,  M.  A. 

37 

Flower,  R.  A. 

110 

138 


AUTHOR  INVLK 


Author 

Page 

Number 

Author 

Page 

Number 

Fogel,  A. 

A9 

Vladdad,  A.  H. 

31 

Fontana,  R.  J. 

92 

Hajek,  B.  E. 

55 

Francis,  J.  T. 

113 

Hammer,  A. 

26 

Frlediander,  U. 

111 

Harger,  R.  0. 

50 

Fries,  R.  W. 

100 

Hartmann,  C.  R.  P. 

9A,98 

Fritchman,  B.  U. 

87 

Hashimoto,  T. 

7A 

Fu,  K.  S. 

8A 

Haskell,  B.  G. 

31 

Fugier-Garrel , C. 

85 

Hayaski,  A. 

73 

Fukunaga,  K. 

82 

Heilman,  M.  E. 

A1 

Gallager,  R.  G. 

58 

Herlestam,  T. 

98 

Geist,  J.  M. 

69 

Herzberg,  M. 

80 

Gersho,  A. 

71 

Hibey,  J.  L. 

31 

Ghani,  F. 

113 

Hirasawa,  S. 

2A 

Gill,  J. 

58 

Ho,  Y.  C. 

77 

Giordano,  A. 

78 

Honorton,  C. 

128 

Gish,  H. 

71 

Housewright,  K.  B. 

36 

Gitlin,  R.  D. 

AO 

Hsu,  F.  M. 

78 

Goldberg,  B.  G. 

119 

Huang,  S.  T. 

75 

Go  Id  fe  in,  11.  D. 

86 

Huang,  T.  S. 

32 

Gotoda,  T. 

106 

Humbler,  P.  A. 

57 

Goulet,  R.  Y. 

68 

Hwang,  T.  Y. 

98 

Gray,  R.  M. 

73,108,126 

Hwang,  W.  G. 

133 

Gualtierotti , A. 

75 

Imber,  Y. 

86 

liaccoun,  D. 

68 

Ingemarsson,  I. 

AA 

139 


AUTHOR  INPEX 


Page  Page 


Author 

Number 

Author 

Number 

Jaynes,  E.  T. 

10,77 

Lee,  D.  T. 

111 

Johannesson,  R. 

69 

Lee,  S.  C. 

48 

Kadota,  T.  T. 

49 

Lempel , A. 

91 

Kailath,  T. 

111 

Leon-Garcia,  A. 

90 

Kanal,  L.  N. 

84 

Leighton,  W.  J.,  Ill 

54 

Kanefsky,  M. 

132 

Leung-Yan-Cheong , S.  R. 

56 

Ranter,  I. 

116 

Lewis,  D.  C. 

85 

Kasaml,  T. 

36 

Lim,  T.  L. 

63 

Rasahara,  K. 

24 

Lin,  S. 

36 

Rashyap,  R.  L. 

105,26 

Linde,  J. 

73 

Rassam,  S.  A. 

63 

Ljung,  L. 

38 

Raveh,  M. 

133 

Longo , G . 

57 

Rawas-Raleh,  G. 

39 

Lu , S . C . 

45 

Razakos,  U. 

27,28,90 

Lu,  S.  Y. 

84 

Riselsteln,  A. 

26 

Lugannanl,  R. 

30 

Roplowitz,  J. 

64 

McGarty,  T.  P. 

80 

Rom,  I. 

80 

Machado  Mata,  J.  A. 

51 

RiJrner,  J. 

107 

Mackenthun,  M. 

90 

Rrlch,  S.  I. 

79 

Maestrlni,  P. 

131 

Rriz,  T.  A. 

93 

Marton,  R. 

107 

aJiLz,  B.  D. 

132 

Masry,  E. 

75,76 

Rurz,  L. 

47,83,89 

Massey,  L, 

107 

Landgrebe,  D.  A. 

28 

Mattsson,  S. 

103 

Le  Chevalier,  F. 

85 

Merkle,  R.  C. 

41 

140 


AUTHOR  JNVEK 


Author 

Page 

Number 

Author 

Page 

Number 

Metzner,  J. 

87 

Nussbaumer,  H. 

60 

Middleton,  D. 

81 

Ogura,  H. 

100 

Mi Is te in,  L.  B. 

38 

Omura,  J.  K. 

36 

Mixsell,  J,  C. 

87 

Ornstein,  D.  S. 

108 

Modes tino,  J.  W. 

100,118 

Osawa , H . 

106 

Mohan , S . 

lOA 

Paaske,  E. 

69 

Mohwinkel,  C. 

89 

Panayircl,  E. 

116 

Morf,  M. 

60,111 

Pankowskl,  B.  J. 

42 

Morgera,  S.  D. 

82,93 

Pao,  Y.  H. 

112 

Morley,  R.  E. 

51 

Papantoni-Kazakos , T. 

90 

Mulder,  L. 

58 

Penm,  H.  W. 

132 

Myung,  N.  -S. 

52 

Penrod,  C.  S. 

63 

Hamekawa,  T. 

24 

Picinbono,  B. 

47 

Negin,  M. 

46 

Pickholtz,  R.  L. 

121 

Nettleton,  R.  W. 

122 

Pieper,  J.  F. 

130 

Neuhoff,  D.  L. 

90,109,125 

Pless,  V. 

22 

Newman,  C.  M. 

100 

Pohllg,  S.  C. 

41 

Nguyen,  M.  K. 

121 

Post,  K.  A. 

130 

Nichols,  H.  E. 

115 

Pradhan,  D.  K. 

94 

Nickel,  U. 

116 

Proakis,  J.  G, 

130 

Nlederrelter , H.  G. 

20,135 

Promhouse,  G. 

24 

Ningo,  A. 

118 

Pupolln,  S. 

52 

Nolte,  L.  W. 

48 

Pursley,  M.  B. 

55,90 

Rand,  R.  H. 

85 

lAl 


AUTHOR  INDEX 


Author 

Page 

Number 

Author 

Page 

Number 

Rao,  T.  R.  N. 

96 

Sgarro,  A. 

57 

Reed,  R.  R. 

130 

Shamir,  A. 

41 

Reisenfeld,  S. 

117 

Shamos , M. 

62 

Rivest,  R.  L. 

41 

Shapiro,  J.  H. 

50,51 

Robinson,  S.  R. 

30 

Sharma,  B.  D.  K. 

31 

Roraa  in , D . H . 

49 

Shedd,  D.  A. 

97 

Root,  W.  L. 

89 

Shields,  P.  C. 

109,135 

Rorabaugh,  C.  B.  , 

Jr. 

46 

Short,  R.  D. 

82 

Rothstein,  J. 

43 

Sidhu,  G.  S. 

110 

Rubin,  I. 

123 

Simionescu,  C.  L. 

125,126 

Rudolph,  L.  D. 

94,98 

Slepian,  D. 

81,100 

Rutledge,  R.  A. 

86 

Sloane,  N.  J.  A, 

23 

Sakrison,  D.  J. 

33,125 

Synder,  D.  L. 

31,51 

Salz,  J. 

38 

Solomonoff,  R.  J. 

129 

Sarwate,  D,  V. 

97 

Spltzer,  F. 

136 

Savage,  J. 

86 

Srihari,  S.  N. 

29 

Schalkwijk,  J.  P. 

M. 

130 

Starks,  S. 

83 

Schneider,  K.  S. 

80 

Stephens,  L. 

32 

Scholtz,  R.  A. 

68,130 

Stockman,  G.  C. 

84 

Schultz,  U.  L. 

112 

Stuck,  B,  W. 

100 

Schwartz,  M. 

120 

Sugiyama,  Y. 

24 

Schwartz,  S.  C. 

49 

Sundberg,  C.  E. 

104,114 

Segall,  A. 

76,121 

Szepanskl,  W. 

27 

Seguin,  G. 

109 

Tam,  L.  C.  T. 

68 

1A2 


AUTHOR  IMPEX 


Page 


Author 

Numbe 

Tan,  H.  H. 

54,127 

Tavares,  S.  E. 

24 

Taylor,  D.  P. 

67 

Tazaki,  S. 

106 

Thomson,  D.  J. 

64 

Thomas , G . 

72 

Thulaslraman , K. 

97 

Tufts,  D.  W. 

113 

Tung,  S.  Y. 

34 

Tyan,  S. 

66 

Ungerboeck,  G. 

115 

VandeLlnde,  V.  D. 

48 

VanSchuppen,  J.  H. 

31 

Venkataraman , K.  N. 

97 

Vieira,  A. 

111 

Vinck,  A.  J. 

130 

VonKaenel,  P.  A, 

22 

Wagner,  T.  J. 

28,63 

Wah,  P.  K.  S. 

72 

Weinert,  H.  L. 

110 

Page 

Author  Number 


Weinstein,  S.  B. 

40 

Welch,  L.  R. 

130 

Weldon,  E.  J. , Jr. 

36 

Whisitt,  S.  J. 

28 

Willett,  M. 

25 

Wilson,  S.  G. 

103 

Wi nograd,  S. 

60 

Witsenhausen , H.  W. 

135 

Wolf,  D. 

105 

Wolf,  J.  K. 

37,105 

Yamada,  Y. 

106 

Yamamura,  S. 

36 

Yan,  J.  K. 

33 

Yao,  A.  C. 

61 

Yao,  K. 

112 

Yeh,  W.  Y, 

131 

Yoon,  C.  S, 

83 

Yuen,  H.  P. 

50,51 

Yum,  T.  S. 

120 

Ziv,  J. 

91 

143 


